-- Logs begin at Sat 2022-06-04 08:25:58 UTC, end at Sat 2022-06-04 09:51:13 UTC. -- Jun 04 09:03:21.583770 dv-cinder-844695-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 04 09:03:21.593321 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jun 4 09:03:21 2022] *** Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: nodename: dv-cinder-844695-1-iscsi Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: machine: x86_64 Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: clock source: unix Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: pcre jit disabled Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: detected number of CPU cores: 4 Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: current working directory: / Jun 04 09:03:21.595318 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: detected binary path: /usr/bin/uwsgi-core Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: your processes number limit is 47680 Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: your memory page size is 4096 bytes Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: detected max file descriptor number: 2048 Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: lock engine: pthread robust mutexes Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: thunder lock: enabled Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 04 09:03:21.595863 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 04 09:03:21.641321 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: Python main interpreter initialized at 0x5651a9d639c0 Jun 04 09:03:21.641321 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: python threads support enabled Jun 04 09:03:21.641321 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: your server socket listen backlog is limited to 100 connections Jun 04 09:03:21.641321 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: your mercy for graceful operations on workers is 90 seconds Jun 04 09:03:21.641709 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: mapped 403077 bytes (393 KB) for 2 cores Jun 04 09:03:21.641786 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: *** Operational MODE: preforking *** Jun 04 09:03:21.642052 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: *** uWSGI is running in multiple interpreter mode *** Jun 04 09:03:21.642052 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: spawned uWSGI master process (pid: 80578) Jun 04 09:03:21.642264 dv-cinder-844695-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 04 09:03:21.646429 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: spawned uWSGI worker 1 (pid: 80589, cores: 1) Jun 04 09:03:21.646429 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: spawned uWSGI worker 2 (pid: 80590, cores: 1) Jun 04 09:03:21.646429 dv-cinder-844695-1-iscsi devstack@keystone.service[80578]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 04 09:03:27.374062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 04 09:03:27.374062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: from cryptography.utils import int_from_bytes Jun 04 09:03:27.374872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 04 09:03:27.374872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: from cryptography.utils import int_from_bytes Jun 04 09:03:27.374986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 04 09:03:27.374986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: from cryptography.utils import int_from_bytes Jun 04 09:03:27.375397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 04 09:03:27.375397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: from cryptography.utils import int_from_bytes Jun 04 09:03:27.424456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 04 09:03:27.424664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 04 09:03:27.424754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 04 09:03:27.424845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] command line args: [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 04 09:03:27.424956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 04 09:03:27.425073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 04 09:03:27.425163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] command line args: [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 04 09:03:27.425232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ================================================================================ {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 04 09:03:27.425416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 04 09:03:27.425509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] admin_token = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.425582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ================================================================================ {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 04 09:03:27.425734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] config_dir = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.425890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] admin_token = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.425974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] config_source = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] config_dir = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] debug = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] config_source = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] debug = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.426921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.427957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_config_append = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_dir = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_config_append = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_file = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_dir = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.428949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_file = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.429954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.430884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.431834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] publish_errors = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] publish_errors = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.432934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] standard_threads = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.433921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] standard_threads = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] use_journal = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] use_json = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] use_stderr = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] use_journal = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.434960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] use_syslog = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.435060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] use_json = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.435186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.435272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] use_stderr = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.435374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] use_syslog = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.435507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.435672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 04 09:03:27.436324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.436471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.436562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.436716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.436831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.436984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.437949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.438918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.439945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.440946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.441942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442624 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.442859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.443966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.444877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.445915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.446963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.447899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.448875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.449945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.450945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.451897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.452863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.453966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.454990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.455937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.456927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.457993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.458989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.459922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.external = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.460915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.external = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.password = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] auth.token = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.461883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.password = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] auth.token = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.462984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.463936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464887 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.464965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.465918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.466973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.467937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.468884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.469849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.470969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.471932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.472170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.473297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.473564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.473830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.474902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.475907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.476945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.477901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.478872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.479991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.480962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.481976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.482145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.482231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.482925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.483964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.484937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.485868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.486979 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.487977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.488960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.489908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.490843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.491955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.492041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.492196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.492275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.492712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.492790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.492963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.493997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.494942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.495916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] role.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] role.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.496885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] role.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] role.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.497838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.498067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.498680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.498852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.498932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499451 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.499920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.500925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.501094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.501283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.501362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.501520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.501607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.501769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.502978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.503971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.504943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505718 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.505959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.506941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.507868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.508882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.509997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.connection = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510567 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.510993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.connection = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.511920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.512901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.513961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.514731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 04 09:03:27.514824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.515071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.515309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 04 09:03:27.515479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 04 09:03:27.543693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.543841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.544065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.544151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.544333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.544570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.553124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.553630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.582010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.582275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.582365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.582567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.641071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.641281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.691542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.691775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.691972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.692214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.759976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.760237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.760479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.760576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.760699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.760807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.760921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.761010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.761138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.761222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.761355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.761574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.785356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.785581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.785665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.785809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.785916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.786887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.798788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.798926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.811749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.811914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.811983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.812979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.813055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.813129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.813241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 04 09:03:27.830553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.830727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.830810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 04 09:03:27.830889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 04 09:03:27.831087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.831173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.831276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 04 09:03:27.831382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 04 09:03:27.831544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.831661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.831803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.831929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.832061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.832190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.832394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.832479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.843832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.843968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.844106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.844248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.844398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.844544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.844680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.844826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.853274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.853444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.856601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.856730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.856880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.856969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.857161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.857246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.857497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.857657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.857783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.857940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.858149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.858226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.858804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.858882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.859975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.887067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.888770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.888877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.889051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.890453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.890741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.894460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.894734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.895013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.896218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.896523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.896813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.902421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.902710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.904200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.905064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.907794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.908079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.908385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.908666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.908939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.909219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.909489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.909612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.909869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.909952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.910182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.910466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.910748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.911030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.911304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.911690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.929290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.929569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.931078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.931368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.934188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.936058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.937998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.938289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.939878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.940185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.943344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.945377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.947044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.949046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.950711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.951002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.951314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.951594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.951878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.952153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.952781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.953087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.953375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.953663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.953946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.954241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.971969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.972255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.972547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.972809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.974140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.974438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.974708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.974977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.980813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.983637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.984796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.985078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.987153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.987432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.988216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.988535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.988814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.989099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.989377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.990564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:27.990861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.991152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.991437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:27.991728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.003550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.003750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.003974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.004258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.004567 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.004845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.005130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.005661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.005958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.006248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.006532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.006829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.007108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.007400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.023912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.026125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.027247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.030046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.031384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.033021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.033613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.034662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.034938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.035221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.035316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.036917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.037201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.037491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.042603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.044211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.044915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.046607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.047642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.047945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.048204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.048509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.050042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.050291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.050582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.050863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.057599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.057889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.058167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.060133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.060450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.060785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.066197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.066478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.066754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.067112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.067452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.067589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.067823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.068090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.068413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.068683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.068992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 04 09:03:28.069088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.069483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.069572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.069649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.069723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.069798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.069871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.070008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.070283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.070560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.070852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.071134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.071422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.071697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.071962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.072248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.072536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 04 09:03:28.108586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5651a9d639c0 pid: 80589 (default app) Jun 04 09:03:28.111757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5651a9d639c0 pid: 80590 (default app) Jun 04 09:03:28.112145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19e4a5-94b9-4d1e-b095-1ec8bfaa637f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:28.112464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19e4a5-94b9-4d1e-b095-1ec8bfaa637f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:28.112742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19e4a5-94b9-4d1e-b095-1ec8bfaa637f None None] PATH_INFO: `/v3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:28.116948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Sat Jun 4 09:03:28 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:37.119814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:37.120354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:37.120442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:37.124739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:03:37.130003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:37.130420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:37.130562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:37.198205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: INFO dbcounter [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] Registered counter for database keystone Jun 04 09:03:37.220933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG oslo_db.sqlalchemy.engines [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80589) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 04 09:03:37.226846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writer thread running {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 04 09:03:37.506597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG passlib.handlers.bcrypt [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80589) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 04 09:03:37.507014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG passlib.handlers.bcrypt [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80589) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 04 09:03:37.562390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:37.563993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:37.689359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 04 09:03:37.863647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:37.932492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: INFO dbcounter [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] Registered counter for database keystone Jun 04 09:03:37.954212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG oslo_db.sqlalchemy.engines [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80590) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 04 09:03:37.960413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writer thread running {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 04 09:03:38.188817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3b3f4-e5d9-4e19-8459-87dbb0fa878d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:38.193291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:38.193964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:38.194251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:38.194532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:38.203488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:38.254923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 04 09:03:38.255273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856afdb-5604-497b-8b4a-c982c3fc4a71 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:38.265970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Sat Jun 4 09:03:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:03:38.272012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:38.286674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291f8a6-33b4-4c3d-8aab-2784e0e8bfe4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:38.291122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:38.292334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:38.292621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:38.292894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] PATH_INFO: `/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:38.294403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] RBAC: Authorizing `identity:get_project(project_id=fbd4724a5a83452ab3b4b7b3823eccc9)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:38.345422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 04 09:03:38.345759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c0dae-8342-4ad3-b49c-e42d7b1719bc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:38.351846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Sat Jun 4 09:03:38 2022] GET /identity/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:40.584330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea8d77-59d3-4b25-88cc-8443feb84df0 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:40.584824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea8d77-59d3-4b25-88cc-8443feb84df0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:40.584824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea8d77-59d3-4b25-88cc-8443feb84df0 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:40.588965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:03:40.595226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:40.595670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:40.595787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:40.616535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:40.618342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:40.626461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 04 09:03:40.648642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea8d77-59d3-4b25-88cc-8443feb84df0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:40.661384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea8d77-59d3-4b25-88cc-8443feb84df0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:40.665561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6bac-9b9d-40e5-ab9c-b3ad4460398e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:40.666187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6bac-9b9d-40e5-ab9c-b3ad4460398e admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:40.666470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6bac-9b9d-40e5-ab9c-b3ad4460398e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:40.666744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6bac-9b9d-40e5-ab9c-b3ad4460398e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:40.675042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab6bac-9b9d-40e5-ab9c-b3ad4460398e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:40.676840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab6bac-9b9d-40e5-ab9c-b3ad4460398e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:40.685480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Sat Jun 4 09:03:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:03:40.691057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:40.700160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40891a-b037-4ea8-bf86-7fcfc088315a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:40.704661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5a898-16c5-4d57-8bfd-162e6dd59f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:40.705526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5a898-16c5-4d57-8bfd-162e6dd59f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:40.705813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5a898-16c5-4d57-8bfd-162e6dd59f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:40.706093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5a898-16c5-4d57-8bfd-162e6dd59f01 admin admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:40.707487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5a898-16c5-4d57-8bfd-162e6dd59f01 admin admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:40.709360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5a898-16c5-4d57-8bfd-162e6dd59f01 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:40.710950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Sat Jun 4 09:03:40 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:43.007994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa48b7-cd24-4362-95dc-8c86a2c989e5 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.008537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa48b7-cd24-4362-95dc-8c86a2c989e5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.008621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa48b7-cd24-4362-95dc-8c86a2c989e5 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.010122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:43.015882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.016360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.016540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.020173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b4704-f2c2-462a-bc9b-0713d1316fb7 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.020497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b4704-f2c2-462a-bc9b-0713d1316fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.020778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b4704-f2c2-462a-bc9b-0713d1316fb7 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.022059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:03:43.027165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.027454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.027732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.033827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:43.035028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:43.066826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG passlib.handlers.bcrypt [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80590) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 04 09:03:43.067251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG passlib.handlers.bcrypt [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80590) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 04 09:03:43.114380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:43.115599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:43.116691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:43.139496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.163469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.218230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:03:43.240088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.242672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d99df-ba03-4531-88d2-8ec47785daa0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.277250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.278114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.278396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.278673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.291311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:43.302929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:43.304006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:43.305031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:03:43.310596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:03:43.311632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.347016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2376c1-77d9-4632-8a7a-0088c8610f10 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.349327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.385654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.386589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.386877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.387157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.397150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.414874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff830d-eb27-47f0-a47a-ec94630ad4d4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.448795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.449461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.449748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.450027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.450775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:43.451085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.453196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.455276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.456827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.465059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:43.470692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.554673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:03:43.560344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.563770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.578410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05730b6-44a3-4d2b-8f7c-86ae397066a4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.612113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.613040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.613326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.613611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.614825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.616313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.618217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:43.624152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.640237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.667043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7a1ca-d13a-4d0a-b843-d4291461fbec None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.700766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.702770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.702770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.702770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.720807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.722400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.724405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa868de-e2db-4c0e-89eb-69c5855933cc None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:03:43.744146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:43.745082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:43.745894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:03:43.745894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:03:43.745894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:03:43.745894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:03:43.745894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 9/18] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:03:43.751738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.760089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.760912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.761204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.761470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.762640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.764121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.765786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 10/19] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:43.771552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:43.808482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.851101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:43.851816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6870812-6f9d-49ba-9f13-65ee87f054c5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.870261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f59e66-ded2-4663-b16d-19b5b6758f8e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:43.886168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.886827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.887107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.887383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.888102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:43.888449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.889867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad460229-6f86-4d13-8994-64529e7aac90 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.904884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:43.905537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:43.905821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:43.906099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:43.906789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 10/20] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:43.906931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.domain.id=default, user.id=8d4e8588f0b448199c8c5e35afe88ad4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:43.907265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:43.908748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32c95-bb39-45a5-826f-bd7a8ed8ac89 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:43.916207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 11/21] 10.222.0.46 () {56 vars in 1322 bytes} [Sat Jun 4 09:03:43 2022] GET /identity/v3/role_assignments?user.id=8d4e8588f0b448199c8c5e35afe88ad4&scope.domain.id=default&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:46.636683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cc25-bb74-41d3-a7a1-f158be1e6621 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:46.637690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cc25-bb74-41d3-a7a1-f158be1e6621 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:46.638219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cc25-bb74-41d3-a7a1-f158be1e6621 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:46.640504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 11/22] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:46.647650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad32bb-efe4-42e0-939a-3874da1011d0 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:46.648747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad32bb-efe4-42e0-939a-3874da1011d0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:46.649271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad32bb-efe4-42e0-939a-3874da1011d0 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:46.650948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:46.651475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:46.651939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:46.655507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 12/23] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:46.662315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:46.664174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:46.664744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:46.695229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:46.696947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:46.707465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:46.709819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:46.869410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 12/24] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:46.878764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 13/25] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:03:46.911360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:46.914687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.014695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:47.056079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:47.092875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d1ad0-89b0-4e36-8c8c-0eacc3c427bd None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.137635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.138803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.139323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.139794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.141596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bf687-20d8-413b-b064-11bdbd2a8564 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.160405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:47.162265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:03:47.191031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:47.191894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:47.192723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:03:47.192723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:03:47.192723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:03:47.192723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:03:47.194706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 14/26] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:03:47.200688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.209027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d8fab-6bfd-4d83-8f93-58213d3f683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.209849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8fab-6bfd-4d83-8f93-58213d3f683d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.210136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8fab-6bfd-4d83-8f93-58213d3f683d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.210414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8fab-6bfd-4d83-8f93-58213d3f683d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.229450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d8fab-6bfd-4d83-8f93-58213d3f683d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e43d8fab-6bfd-4d83-8f93-58213d3f683d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:47.251434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:47.252379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:03:47.253142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 13/27] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:03:47.253982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608548e-11ee-4982-bab4-a1f7c2c7d8bb None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.254410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608548e-11ee-4982-bab4-a1f7c2c7d8bb None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.254841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608548e-11ee-4982-bab4-a1f7c2c7d8bb None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.269638 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 14/28] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:47 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:47.270684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608548e-11ee-4982-bab4-a1f7c2c7d8bb None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.317142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88b5f0-be44-4ee0-9ec3-7f0968325f56 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.345094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:47.354038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.354966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.355510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.355949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.356971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:47.361062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:47.361062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940f2d-c28f-4904-8917-da99250c44f8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:47.379225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 15/29] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:47.380880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.381395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.381908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.399641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:47.401354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:47.415305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608548e-11ee-4982-bab4-a1f7c2c7d8bb None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.423886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:47.454008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.454919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.455348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.455773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.456856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:47.457426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:47.459012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb902ec1-1aad-465e-9dc9-2ba7bfeeccc4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:47.473058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 15/30] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:47.474865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f665-81a9-400e-95cb-e832f34300a0 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.475358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f665-81a9-400e-95cb-e832f34300a0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.475838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f665-81a9-400e-95cb-e832f34300a0 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.481523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 16/31] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:47.482584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f665-81a9-400e-95cb-e832f34300a0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.482716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 16/32] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:47.484407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.598134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f74411-f485-4c76-b6b2-e6c42317a7b4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.616980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:47.621392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f665-81a9-400e-95cb-e832f34300a0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.638005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe194b69-d0ff-42c1-b99f-5d790d8ab034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.639797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194b69-d0ff-42c1-b99f-5d790d8ab034 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.640232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194b69-d0ff-42c1-b99f-5d790d8ab034 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.649639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194b69-d0ff-42c1-b99f-5d790d8ab034 None admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.649639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe194b69-d0ff-42c1-b99f-5d790d8ab034 None admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:47.649639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe194b69-d0ff-42c1-b99f-5d790d8ab034 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:47.649639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 17/33] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:03:47 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:47.649639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.649639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.650562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.665677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.667960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.668519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.668984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.669731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:47.670517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:47.671003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:47.672599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:47.752253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 18/34] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:47.753025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.869415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938b068-6b20-4937-9664-387a16bd3c27 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:47.896685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:47.907308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a406b7-0a25-4e94-a441-ff2488258a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:47.907985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a406b7-0a25-4e94-a441-ff2488258a1a None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:47.908349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a406b7-0a25-4e94-a441-ff2488258a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:47.908637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a406b7-0a25-4e94-a441-ff2488258a1a None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:47.909488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a406b7-0a25-4e94-a441-ff2488258a1a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:47.911037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a406b7-0a25-4e94-a441-ff2488258a1a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:47.924008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 17/35] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:03:47 2022] POST /identity/v3/projects => generated 301 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:03:47.925466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:47.957230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:48.056030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03462cb8-aac9-444d-be99-0ac0342d2d33 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:48.101083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:48.101950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.102237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.102517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.103913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:48.105598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:48.107637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 18/36] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:48.108256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:48.175334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 19/37] 10.222.0.46 () {60 vars in 1131 bytes} [Sat Jun 4 09:03:47 2022] POST /identity/v3/roles => generated 219 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:03:48.177352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019802ef-26b5-436e-887f-936affcb8eb4 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.177856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019802ef-26b5-436e-887f-936affcb8eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.178411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019802ef-26b5-436e-887f-936affcb8eb4 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.180047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 20/38] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:48.181163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019802ef-26b5-436e-887f-936affcb8eb4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:48.220821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:48.230066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740590a-bab4-4b6c-99ba-0d7eb390855a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:48.275101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bc075-31b4-449f-b6e7-e45efc910182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:48.275835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bc075-31b4-449f-b6e7-e45efc910182 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.276122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bc075-31b4-449f-b6e7-e45efc910182 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.276450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bc075-31b4-449f-b6e7-e45efc910182 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.277232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bc075-31b4-449f-b6e7-e45efc910182 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:48.279420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bc075-31b4-449f-b6e7-e45efc910182 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:48.309397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019802ef-26b5-436e-887f-936affcb8eb4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:48.346584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4044d-44c8-4d85-9d87-d2362a647e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:48.347507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4044d-44c8-4d85-9d87-d2362a647e7f None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.347847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4044d-44c8-4d85-9d87-d2362a647e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.348134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4044d-44c8-4d85-9d87-d2362a647e7f None admin] PATH_INFO: `/v3/domains/default/users/8d4e8588f0b448199c8c5e35afe88ad4/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.350512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4044d-44c8-4d85-9d87-d2362a647e7f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8d4e8588f0b448199c8c5e35afe88ad4, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:48.352178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4044d-44c8-4d85-9d87-d2362a647e7f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:48.359245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 19/39] 10.222.0.46 () {60 vars in 1131 bytes} [Sat Jun 4 09:03:48 2022] POST /identity/v3/roles => generated 225 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:03:48.360918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0de04-e4eb-4d29-877a-a6af2323a24d None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.361220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0de04-e4eb-4d29-877a-a6af2323a24d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.361511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0de04-e4eb-4d29-877a-a6af2323a24d None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.363619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 20/40] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:48.364337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.364606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.368776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.387753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:48.390603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:48.456109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 21/41] 10.222.0.46 () {58 vars in 1361 bytes} [Sat Jun 4 09:03:48 2022] PUT /identity/v3/domains/default/users/8d4e8588f0b448199c8c5e35afe88ad4/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:03:48.462016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.462390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.462743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.483505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 21/42] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:48.494202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:48.496439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:48.519141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:48.537672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:48.606210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 22/43] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:48.620185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:48.630805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:48.666255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180c77f-1685-4597-ac97-40fb61570b63 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:48.709441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8e17f-22cb-4b35-9e16-03e5ac094c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:48.710768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8e17f-22cb-4b35-9e16-03e5ac094c45 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.710768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8e17f-22cb-4b35-9e16-03e5ac094c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.710928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8e17f-22cb-4b35-9e16-03e5ac094c45 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.711650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8e17f-22cb-4b35-9e16-03e5ac094c45 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:48.713092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8e17f-22cb-4b35-9e16-03e5ac094c45 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:48.749798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae3907-1468-4f82-9891-9fe19d978313 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:48.784957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:48.785747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.786028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.786305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.787463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:48.788956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:48.790580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 23/44] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:48.796754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:48.826627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:48.880529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 22/45] 10.222.0.46 () {60 vars in 1131 bytes} [Sat Jun 4 09:03:48 2022] POST /identity/v3/roles => generated 223 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:03:48.897213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c081bf-154d-4dd1-b32f-f2a17aec1f50 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:48.939921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c019ab9-ea94-4221-a4b4-514261b9fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:48.940642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019ab9-ea94-4221-a4b4-514261b9fc95 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:48.940933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019ab9-ea94-4221-a4b4-514261b9fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:48.941215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019ab9-ea94-4221-a4b4-514261b9fc95 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:48.942050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c019ab9-ea94-4221-a4b4-514261b9fc95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:48.943579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c019ab9-ea94-4221-a4b4-514261b9fc95 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:49.055407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 24/46] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:03:48 2022] POST /identity/v3/projects => generated 312 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:03:50.744854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b2e9b-919e-4549-886b-0a2964eb8a47 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:50.745375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b2e9b-919e-4549-886b-0a2964eb8a47 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:50.745458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b2e9b-919e-4549-886b-0a2964eb8a47 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:50.746811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 23/47] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:50.752826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:50.753249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:50.760716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:50.771684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:50.772977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:50.841401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 25/48] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:03:50.865550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b2e9b-919e-4549-886b-0a2964eb8a47 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:50.968958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b2e9b-919e-4549-886b-0a2964eb8a47 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.005442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.006167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.006450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.006728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.013648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:51.017548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:51.025327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:51.026179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:03:51.027083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 24/49] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:03:51.032936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:51.047761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:51.129820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b658c-9b85-404b-9b3c-3906f3637814 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.163439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.164111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.164426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.164710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.165461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:51.165768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:51.167177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051be36-d43f-4fec-bedd-8f961ce3b0f6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:51.176350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 26/50] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:51.181900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:51.284767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a988db-5a69-41ae-bedb-0a9a0f4ed776 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.295356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae715d9-8074-493b-8f06-b8583bf7648a None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.296190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae715d9-8074-493b-8f06-b8583bf7648a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.296495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae715d9-8074-493b-8f06-b8583bf7648a None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.297867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 27/51] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:03:51.302996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.303357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.303708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.320677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:51.320943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.321829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.321939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:51.322112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.322387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.323812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:51.325307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:51.326950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 25/52] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:03:51 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:51.332229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:51.388120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 28/53] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:03:51.398072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:51.411121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:51.419132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:51.432603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcc118-a1e4-489b-8391-05ed97dff386 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.466540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.467568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.467853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.468134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.469512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:51.470992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:51.472767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 26/54] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:51.478699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:51.510421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec58d92-3c7f-4faf-8ac4-6f85ad871bd9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.544423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.545156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.545970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.546255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.547439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:51.548933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:51.550563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 29/55] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:51.555944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:51.580125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8193281-14da-423a-b2b3-57a388db7163 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.613831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.614495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.614782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.615066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.615897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.domain.id=default, user.id=8d4e8588f0b448199c8c5e35afe88ad4) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:51.616211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:51.617686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0ca6a-cc10-4ff4-80f4-c556c4fe559b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:51.620269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:51.628508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 27/56] 10.222.0.46 () {56 vars in 1322 bytes} [Sat Jun 4 09:03:51 2022] GET /identity/v3/role_assignments?user.id=8d4e8588f0b448199c8c5e35afe88ad4&scope.domain.id=default&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:03:51.631282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:51.661472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a81405-9318-4f81-a176-d16b10a8159d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:51.699016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635aee1e-e1eb-4993-b67f-ccf2a8a9e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:51.699716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aee1e-e1eb-4993-b67f-ccf2a8a9e324 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:51.700015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aee1e-e1eb-4993-b67f-ccf2a8a9e324 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:51.700325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aee1e-e1eb-4993-b67f-ccf2a8a9e324 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:51.701070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635aee1e-e1eb-4993-b67f-ccf2a8a9e324 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:51.702632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635aee1e-e1eb-4993-b67f-ccf2a8a9e324 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:51.832788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 30/57] 10.222.0.46 () {60 vars in 1140 bytes} [Sat Jun 4 09:03:51 2022] POST /identity/v3/projects => generated 298 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:03:54.091418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889a7d-14b1-4d37-991d-a5baa1dc90a8 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:54.091932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889a7d-14b1-4d37-991d-a5baa1dc90a8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:54.092044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889a7d-14b1-4d37-991d-a5baa1dc90a8 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:54.093904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 28/58] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:54.099121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:54.099535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:54.099689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:54.116769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:54.117968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:54.182363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 31/59] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:54.206806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889a7d-14b1-4d37-991d-a5baa1dc90a8 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:54.308206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889a7d-14b1-4d37-991d-a5baa1dc90a8 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:54.341316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69685566-28c3-40ca-a160-bca6e6ae1185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:54.342152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69685566-28c3-40ca-a160-bca6e6ae1185 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:54.343004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69685566-28c3-40ca-a160-bca6e6ae1185 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:54.343313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69685566-28c3-40ca-a160-bca6e6ae1185 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:54.344531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69685566-28c3-40ca-a160-bca6e6ae1185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:54.345989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69685566-28c3-40ca-a160-bca6e6ae1185 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:54.347608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 29/60] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:54.353115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:54.406755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:54.446943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ca1a-fde1-4bf0-813c-3de22b2b3ab3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:54.480733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46377caf-ae22-4d16-acd6-02af9b015a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:54.481394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46377caf-ae22-4d16-acd6-02af9b015a49 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:54.481680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46377caf-ae22-4d16-acd6-02af9b015a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:54.481962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46377caf-ae22-4d16-acd6-02af9b015a49 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:54.482665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46377caf-ae22-4d16-acd6-02af9b015a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:54.484317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46377caf-ae22-4d16-acd6-02af9b015a49 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:54.602225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 32/61] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:03:54 2022] POST /identity/v3/users => generated 275 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:03:57.834231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec40c-9d14-42b7-9bce-99398beb54b9 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:57.834231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec40c-9d14-42b7-9bce-99398beb54b9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:57.834231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec40c-9d14-42b7-9bce-99398beb54b9 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:57.835170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 30/62] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:57.839750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:57.840552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:57.841007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:57.862645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:57.864123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:57.907089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a86e0a-efdb-40fc-aa39-1c57ac01b305 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:57.907606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a86e0a-efdb-40fc-aa39-1c57ac01b305 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:57.908057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a86e0a-efdb-40fc-aa39-1c57ac01b305 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:57.909662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 31/63] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:03:57.915749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:57.915749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:57.915749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:57.943122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:57.944366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:57.976685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 33/64] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:58.008815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.049401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:58.054974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 32/65] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:03:58.086647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.098035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:58.197612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0853dc9-e52e-4fb6-aa6c-136f599d45bc None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.266099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.266939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.267229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.267512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.282693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2e107-332e-43eb-9c8a-af055761633f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.299118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:58.307543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:58.308557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:03:58.309339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:03:58.312535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 34/66] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:03:58.318613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.358145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.359852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.360301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.360750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.396506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:58.414330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:58.416153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:03:58.417815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:03:58.422961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 33/67] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:03:58.431788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.502815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:58.549281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a577816-248b-413d-9f05-61d67c03d46c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.551251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a9cc-b073-493c-ae4a-2fe2e571abc7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.551393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:58.586735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.587269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.587709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.588121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.588243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.588471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.588692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.588977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.589704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:58.589823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:58.590025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:58.590413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:58.591421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dba02-efab-47f1-a881-0ad3f2f25106 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:58.592121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:58.602264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 34/69] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:58.602466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 35/69] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:58.603888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423444-5879-4eaf-8816-9c8392f73742 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.604180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423444-5879-4eaf-8816-9c8392f73742 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.604486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423444-5879-4eaf-8816-9c8392f73742 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.605757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 36/70] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:58.609964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.611339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423444-5879-4eaf-8816-9c8392f73742 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.736852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423444-5879-4eaf-8816-9c8392f73742 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.740126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588e482-edcf-4998-99bd-cbf10d6ba491 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.758474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:58.775882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350409-c941-4827-8746-b6770cb02088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.778517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350409-c941-4827-8746-b6770cb02088 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.778517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350409-c941-4827-8746-b6770cb02088 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.778517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350409-c941-4827-8746-b6770cb02088 None admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.778881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44350409-c941-4827-8746-b6770cb02088 None admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:58.780010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44350409-c941-4827-8746-b6770cb02088 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:58.782638 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 35/71] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:58.783011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad05e-8761-4036-ab5f-073c192d85f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.783644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7457fcb-5008-4110-af1c-cb08077e49ef None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.783931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7457fcb-5008-4110-af1c-cb08077e49ef None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.784215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7457fcb-5008-4110-af1c-cb08077e49ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.784648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad05e-8761-4036-ab5f-073c192d85f7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.785078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad05e-8761-4036-ab5f-073c192d85f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.785497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad05e-8761-4036-ab5f-073c192d85f7 None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.802671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ad05e-8761-4036-ab5f-073c192d85f7 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:58.802876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c7457fcb-5008-4110-af1c-cb08077e49ef None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:58.804392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c7457fcb-5008-4110-af1c-cb08077e49ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:58.804514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ad05e-8761-4036-ab5f-073c192d85f7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:58.806106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 37/72] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:58.807452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa1e10-ed36-49ad-a4bb-0b9907d944d8 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.807741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa1e10-ed36-49ad-a4bb-0b9907d944d8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.808030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa1e10-ed36-49ad-a4bb-0b9907d944d8 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.814807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 38/73] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:58.814807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa1e10-ed36-49ad-a4bb-0b9907d944d8 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:58.839366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:58.876208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 36/74] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:58.878099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.878391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.878670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.896573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:58.897763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:58.925013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa1e10-ed36-49ad-a4bb-0b9907d944d8 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:58.959357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19d3ca-9d14-46f6-8ee4-c251ace9269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:58.960009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19d3ca-9d14-46f6-8ee4-c251ace9269e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:58.960317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19d3ca-9d14-46f6-8ee4-c251ace9269e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:58.960602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19d3ca-9d14-46f6-8ee4-c251ace9269e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:58.972452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 37/75] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:58.973302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.020584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.031403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19d3ca-9d14-46f6-8ee4-c251ace9269e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.032930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19d3ca-9d14-46f6-8ee4-c251ace9269e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.056880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.074980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef243e-39d7-4bb1-a069-3949a7f9d947 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.111214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.111797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.112135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.112377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.133508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 39/76] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:03:59.134776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d843ca-9bb1-4224-ad22-46defc059d81 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.135077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d843ca-9bb1-4224-ad22-46defc059d81 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.135351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d843ca-9bb1-4224-ad22-46defc059d81 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.136668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 40/77] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:03:59.140511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d843ca-9bb1-4224-ad22-46defc059d81 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.185898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.187929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.233320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.241455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d843ca-9bb1-4224-ad22-46defc059d81 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.267918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.279679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.281658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.281658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.281658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.291698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:59.292524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 38/78] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:03:59.297301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:59.300804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:59.301672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:03:59.302426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 41/79] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:03:59.303189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.401627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da010-8fb5-4c2b-b541-73e052438e41 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.402856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a28c-0351-42fd-8928-1517e42fac7d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.435592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.436148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd0be9-34bb-4983-8fff-6e900b8bbb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.436434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.437252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.437531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.437620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd0be9-34bb-4983-8fff-6e900b8bbb11 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.437819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd0be9-34bb-4983-8fff-6e900b8bbb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.438096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd0be9-34bb-4983-8fff-6e900b8bbb11 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.439283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd0be9-34bb-4983-8fff-6e900b8bbb11 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.440771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd0be9-34bb-4983-8fff-6e900b8bbb11 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.442496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 42/80] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:59.447594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.447594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.447594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.453078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:03:59.461008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:03:59.461871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:03:59.462635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 39/81] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:03:59.463638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:03:59.463721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.464139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:03:59.476226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.511768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.532663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 43/82] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:03:59.533518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.578373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755bc4-eb45-4d39-b8dc-187752be15f8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.617816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.618966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.619249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.619526 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.620826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.622437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.624856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 40/83] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:59.630351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.644710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df7c47-dc3a-4b3a-bd6b-b63b92a1f726 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.681038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.685454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.685454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.685454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.685454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:59.685454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.685454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.693906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 44/84] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:59.694759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.709702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.734976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4ac5a-c9fb-4b2b-9d48-249718cee724 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.739265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.770370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.771033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.771322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.771603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.772482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=8d4e8588f0b448199c8c5e35afe88ad4) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:59.772800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.774258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.779771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 41/85] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/role_assignments?user.id=8d4e8588f0b448199c8c5e35afe88ad4&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:59.780636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.799816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e58f8-ab59-4083-9de5-9ae805e58458 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.842318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.843172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.843473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.843867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.844697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:03:59.845012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.846468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.861234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 45/86] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:59.861234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.911357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859433-5ab1-4b38-84fc-fe5924bf3f40 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.950999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:03:59.951733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:03:59.952032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:03:59.952364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:03:59.953645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:03:59.955118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:03:59.956864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 42/87] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:03:59.957650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:03:59.971189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:03:59.980625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4f27-6727-4336-af9f-e9f4d4882c4e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:03:59.982966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:00.019948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.020638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.020924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.021466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.022040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=7fa620e446a2472186b5f2e19bc28fd4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:00.022351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.023795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.029444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 46/88] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/role_assignments?user.id=7fa620e446a2472186b5f2e19bc28fd4&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=60342ac4b2b144e98d66f95f23780594 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:00.033226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.080242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da737fca-ac83-48f0-92b0-4ed89fc4f6df None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:00.123689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.124719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.125056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.125271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.126594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.128240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.130422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 43/89] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:03:59 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:00.131052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.181793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1765e-b6c6-4194-90f8-2479b351faf7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:00.220206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:00.220796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.221968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.222459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.222955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.224522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.233165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.233165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 47/90] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:00 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:00.233165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.251421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c678985-9a05-474a-9154-f3d3f3a20d9d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:00.269441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:00.314411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.315097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.315388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.315668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.316430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.317907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.353823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594208f-35eb-4fb2-8db4-0c931f75e46c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:00.389128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.390338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.390627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.390906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.412059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 44/91] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:00 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:00.413625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.534555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.535974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.611344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:00.633433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b37ec2-878d-4ec9-992f-1f86e666d7bd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:00.682058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:00.690046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.691103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.691644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.692142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.699733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 48/92] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:00.705994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.785173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.787011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.860259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cd4a8-ef85-49c6-a0ca-363091a9e5b2 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:00.905184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:00.907087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:00.907591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:00.908070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:00.909606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:00.911473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:00.916396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 49/93] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:00 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:00.920174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.925366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:00.950990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 45/94] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:00.957244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:00.994252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:01.047161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29b312-c6ba-4f07-b87d-24e85c052ede None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:01.073247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff9ddb-583b-4cdc-80bb-07c87faae641 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:01.085712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:01.086470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:01.086830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:01.087184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:01.088214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] RBAC: Adding query filter params (role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=7fa620e446a2472186b5f2e19bc28fd4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:01.088633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:01.090170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:01.096926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 50/95] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:00 2022] GET /identity/v3/role_assignments?user.id=7fa620e446a2472186b5f2e19bc28fd4&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:01.115164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6312d-bd7c-4ec9-874e-adb6f84d3b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:01.116532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6312d-bd7c-4ec9-874e-adb6f84d3b1b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:01.117024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6312d-bd7c-4ec9-874e-adb6f84d3b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:01.117525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6312d-bd7c-4ec9-874e-adb6f84d3b1b None admin] PATH_INFO: `/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:01.119408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6312d-bd7c-4ec9-874e-adb6f84d3b1b None admin] RBAC: Authorizing `identity:get_project(project_id=3a08f9f2846a43f8ae40b4be3bb5cbd9)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:01.121175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6312d-bd7c-4ec9-874e-adb6f84d3b1b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:01.123591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 46/96] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:00 2022] GET /identity/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:01.130600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:01.204649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:01.244401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39bf25-c751-499d-92a1-0f1d66e2320e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:01.307043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:01.308022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:01.308546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:01.309010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:01.310074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=3a08f9f2846a43f8ae40b4be3bb5cbd9, user.id=7fa620e446a2472186b5f2e19bc28fd4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:01.310580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:01.313093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc841e-ae14-4fec-8c43-82b82eaa67f9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:01.323640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 51/97] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:01 2022] GET /identity/v3/role_assignments?user.id=7fa620e446a2472186b5f2e19bc28fd4&scope.project.id=3a08f9f2846a43f8ae40b4be3bb5cbd9&role.id=60342ac4b2b144e98d66f95f23780594 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:02.554416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f062e5-84b4-46a4-81c4-ebfc3e0bd0a3 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:02.554896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f062e5-84b4-46a4-81c4-ebfc3e0bd0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:02.554896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f062e5-84b4-46a4-81c4-ebfc3e0bd0a3 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:02.556537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 47/98] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:02.561982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:02.562793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:02.563267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:02.585260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:02.586893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:02.675907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 52/99] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:02.721747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f062e5-84b4-46a4-81c4-ebfc3e0bd0a3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:02.865505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:02.870168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f062e5-84b4-46a4-81c4-ebfc3e0bd0a3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:02.916963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:02.917817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:02.918101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:02.918377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:02.934950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:02.943273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:02.944108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:02.945352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 48/100] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:02.951962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:03.028947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:03.068452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c450-2998-4ac8-b032-9c66570bbeb3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:03.103967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:03.104667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.104948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.105220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.105966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:03.106273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:03.107652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcae69-ec3f-4153-b425-3a5b32e32ea1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:03.116445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 53/101] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:03.125640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:03.200175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1fdeb-608f-46c2-b450-2eae6c8cdb2b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.200646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1fdeb-608f-46c2-b450-2eae6c8cdb2b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.200762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1fdeb-608f-46c2-b450-2eae6c8cdb2b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.202864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 54/102] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:03.208322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.210340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.210340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.228848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:03.231090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:03.266220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5fe79-4c9e-4023-afc6-212080143b8a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:03.338404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:03.339342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.339637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.339921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.342201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:03.343731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:03.346503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 49/103] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:03 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:03.351852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:03.352689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 55/104] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:03.397553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:03.459686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:03.525189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93439a-898d-40b6-a6c3-7672a642524a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:03.568772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:03.606648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:03.607551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.607838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.608117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.616130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:03.617705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:03.622239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 50/105] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:03 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:03.628966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:03.674283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491d4ee-ab8d-45aa-af17-c206eee9fb6c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:03.759849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:03.763355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.763355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.763355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.790911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:03.806855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:03.807757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:03.809010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 56/106] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:03.815699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:03.846000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:03.867820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47026e-02b6-401a-8339-e1fcecc64fca None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:03.961018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4506dc-2d8c-4bf0-baa1-a70b284c3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:03.961954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4506dc-2d8c-4bf0-baa1-a70b284c3508 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:03.962248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4506dc-2d8c-4bf0-baa1-a70b284c3508 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:03.962531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4506dc-2d8c-4bf0-baa1-a70b284c3508 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/8d4e8588f0b448199c8c5e35afe88ad4/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:03.964670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4506dc-2d8c-4bf0-baa1-a70b284c3508 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e98f0415e245278b0f5ad4f985e67c, user_id=8d4e8588f0b448199c8c5e35afe88ad4, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:03.966138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4506dc-2d8c-4bf0-baa1-a70b284c3508 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:03.998531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4225-d8b9-4289-96ee-c36687ae887e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:04.025429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:04.034879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:04.035638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.035932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.036215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.037002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:04.037367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:04.038780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:04.047993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 51/107] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:03 2022] PUT /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/8d4e8588f0b448199c8c5e35afe88ad4/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:04.049534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baf2e6-52a4-4f9b-ac60-408462d4bb74 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.049833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baf2e6-52a4-4f9b-ac60-408462d4bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.050118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baf2e6-52a4-4f9b-ac60-408462d4bb74 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.055110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 52/108] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:04.058535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 57/109] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:04.060740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682c17-431c-42da-8741-0118e34cb72f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.061105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682c17-431c-42da-8741-0118e34cb72f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.061514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682c17-431c-42da-8741-0118e34cb72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.065412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:04.094565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-04682c17-431c-42da-8741-0118e34cb72f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:04.095821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-04682c17-431c-42da-8741-0118e34cb72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:04.193174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4323-23f1-4fd4-9734-0f06f1d870b9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:04.222490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 53/110] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:04.230578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:04.231472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.231763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.232044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.233331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:04.234873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:04.236922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 58/111] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:04 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:04.242212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682c17-431c-42da-8741-0118e34cb72f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:04.271969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:04.376730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:04.377040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:04.395494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682c17-431c-42da-8741-0118e34cb72f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:04.438327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ccc5-cd8a-42d7-acd3-793158604d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:04.439215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ccc5-cd8a-42d7-acd3-793158604d22 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.439503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ccc5-cd8a-42d7-acd3-793158604d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.439783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ccc5-cd8a-42d7-acd3-793158604d22 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.441087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ccc5-cd8a-42d7-acd3-793158604d22 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:04.442625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ccc5-cd8a-42d7-acd3-793158604d22 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:04.444582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 54/112] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:04 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:04.445756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a697fd-5a60-41c7-9a17-3b07deb1d90a None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.446047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a697fd-5a60-41c7-9a17-3b07deb1d90a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.446334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a697fd-5a60-41c7-9a17-3b07deb1d90a None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.449497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 55/113] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:04.453391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a697fd-5a60-41c7-9a17-3b07deb1d90a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:04.461113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4539c-ff24-4c84-a1cf-93885cf008ec None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:04.523942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74264a49-02d0-40bd-9812-72c2577d1b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:04.524691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264a49-02d0-40bd-9812-72c2577d1b59 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.524978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264a49-02d0-40bd-9812-72c2577d1b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.525256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264a49-02d0-40bd-9812-72c2577d1b59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.547662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74264a49-02d0-40bd-9812-72c2577d1b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-74264a49-02d0-40bd-9812-72c2577d1b59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:04.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:04.565530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:04.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 59/114] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:04.575192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.575794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.577051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.609508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:04.610751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:04.643134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a697fd-5a60-41c7-9a17-3b07deb1d90a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:04.706522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814ee30-22e5-462e-9156-9964e30d2b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:04.707487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814ee30-22e5-462e-9156-9964e30d2b2f None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.707823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814ee30-22e5-462e-9156-9964e30d2b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.708112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814ee30-22e5-462e-9156-9964e30d2b2f None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/7fa620e446a2472186b5f2e19bc28fd4/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.710831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814ee30-22e5-462e-9156-9964e30d2b2f None admin] RBAC: Authorizing `identity:create_grant(project_id=53e98f0415e245278b0f5ad4f985e67c, user_id=7fa620e446a2472186b5f2e19bc28fd4, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:04.712442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814ee30-22e5-462e-9156-9964e30d2b2f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:04.734186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 60/115] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:04.735156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:04.771838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:04.797180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 56/116] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:04 2022] PUT /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/7fa620e446a2472186b5f2e19bc28fd4/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:04:04.798706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e88ef-e4e5-456e-a70e-bf479e206b9b None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.799104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e88ef-e4e5-456e-a70e-bf479e206b9b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.799491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e88ef-e4e5-456e-a70e-bf479e206b9b None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.801012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 57/117] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:04.801901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e88ef-e4e5-456e-a70e-bf479e206b9b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:04.856707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:04.868029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af186f-59f4-4bf4-9f48-dd3baa9f8379 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:04.913861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:04.914590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.914876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.915158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.915932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:04.916243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:04.918208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbc845-7b17-43bf-9a8a-737f9bd7afb5 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:04.928683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 61/118] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:04.931801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cd830-fabc-4c85-9640-b22899d6be13 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:04.932404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cd830-fabc-4c85-9640-b22899d6be13 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:04.932910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cd830-fabc-4c85-9640-b22899d6be13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:04.957686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b24cd830-fabc-4c85-9640-b22899d6be13 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:04.959509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b24cd830-fabc-4c85-9640-b22899d6be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:05.011437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e88ef-e4e5-456e-a70e-bf479e206b9b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.053983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b28331-2a57-45e8-9fc5-a8ec4bccc646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.054672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 62/119] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:05.054778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b28331-2a57-45e8-9fc5-a8ec4bccc646 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.054992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b28331-2a57-45e8-9fc5-a8ec4bccc646 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.055848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b28331-2a57-45e8-9fc5-a8ec4bccc646 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.055958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64356e2e-de09-487d-a6ab-731b4aeea276 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.056219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64356e2e-de09-487d-a6ab-731b4aeea276 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.056540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64356e2e-de09-487d-a6ab-731b4aeea276 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.056636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b28331-2a57-45e8-9fc5-a8ec4bccc646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:05.057901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 63/120] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:05.058408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b28331-2a57-45e8-9fc5-a8ec4bccc646 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:05.058664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64356e2e-de09-487d-a6ab-731b4aeea276 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.060404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 58/121] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:05.068193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.068567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.068857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.088105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:05.089580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:05.095957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:05.171623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 59/122] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:05.172757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.191321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:05.219789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64356e2e-de09-487d-a6ab-731b4aeea276 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.265981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.267038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.268082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.268496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] PATH_INFO: `/v3/users/fe8ddf9ca04e49349faf1804f40edfe7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.287806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] RBAC: Authorizing `identity:get_user(user_id=fe8ddf9ca04e49349faf1804f40edfe7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:05.289856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:05.292446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 64/123] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/users/fe8ddf9ca04e49349faf1804f40edfe7 => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:05.293608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.297347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d5c61-00d9-43a5-9414-7114a563fd00 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.344487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.345145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.345437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.345718 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.346549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:05.348019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:05.389508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:05.404881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9856-4163-4a4a-8639-116ab79f6bbd None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.443243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 60/124] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:04:05 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:05.448607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.448607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.448607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.448607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.452635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.469655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:05.488433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:05.489562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:05.490522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:05.494263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 65/125] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:05.496549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.558294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:05.634992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab071ec-e262-4794-a162-d426ec227a93 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.705988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.707676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.709380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.710531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.719803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae7f80-93a8-4d60-a952-fadb8c043dda None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.734081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:05.745119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:05.746095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:05.746869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 61/126] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:05.747514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.771662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.773024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.773322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.773609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.794045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:05.865430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:05.867456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:05.878268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c20da-b852-4158-872b-502dc5a3bc2f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:05.908276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:05.924347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:05.925040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:05.925679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:05.925997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:05.926770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:05.927446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:05.929051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:05.940905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 62/127] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:05.942185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:05.991175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 66/128] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:05.992513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.078536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ca44a-f62c-498f-a8c9-f4c3338dd4ab None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.079161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:06.120275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.120986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.121279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.121563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.122334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:06.122652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.124653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.133915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 63/129] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.134812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.173327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a74c7-ec1e-4ef1-941d-962328fb5fad None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.188462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:06.215501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.218578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.218869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.219146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.223284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.224819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.226703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 67/130] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:05 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.227327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.318090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720cbba-c857-45df-b5bd-5d5c321a773e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.391939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a782a5-4b6f-4bc5-a41a-46172447cb92 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.396487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.397390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.397675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.397953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.400226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.401879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.403716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 64/131] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:06 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.404580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.432354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:06.516409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3129f-f63f-45ae-86d2-cf7fab03909b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.529599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.531190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.531482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.531762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.533012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.534478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.536330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 68/132] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:06 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.536941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.537515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:06.552533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.554116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.554116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.554116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.555259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.557311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.559639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 65/133] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:06 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.560340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.677433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be4a0f-0b2a-4f34-9445-d3b3654c8204 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.715597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.716476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.716813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.717095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] PATH_INFO: `/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.718446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] RBAC: Authorizing `identity:get_project(project_id=3a08f9f2846a43f8ae40b4be3bb5cbd9)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.719912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.721998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 66/134] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:06 2022] GET /identity/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.722601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.785070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6bdb2-23ed-4751-8281-1e5cfd447f8c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.809960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:06.820823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.821488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.821998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.822104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.822876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] RBAC: Adding query filter params (role.id=9fa4a6cb6863453697d834107e908e15, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=fe8ddf9ca04e49349faf1804f40edfe7) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:06.823282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.824661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.831474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 69/135] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:06 2022] GET /identity/v3/role_assignments?user.id=fe8ddf9ca04e49349faf1804f40edfe7&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=9fa4a6cb6863453697d834107e908e15 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:06.831871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:06.870963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eca23-1c4f-4386-a12f-629b628822f5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:06.911100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888b997-589d-4e2c-a2d7-a9ab692aca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:06.912432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b997-589d-4e2c-a2d7-a9ab692aca15 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:06.912900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b997-589d-4e2c-a2d7-a9ab692aca15 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:06.913358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b997-589d-4e2c-a2d7-a9ab692aca15 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/7fa620e446a2472186b5f2e19bc28fd4/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:06.916188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888b997-589d-4e2c-a2d7-a9ab692aca15 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e98f0415e245278b0f5ad4f985e67c, user_id=7fa620e446a2472186b5f2e19bc28fd4, role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:06.917921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888b997-589d-4e2c-a2d7-a9ab692aca15 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:06.948953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:07.001318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 67/136] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:06 2022] PUT /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/7fa620e446a2472186b5f2e19bc28fd4/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:07.003010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516421-8b70-4d4c-9dc1-05c92dd07425 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.003512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516421-8b70-4d4c-9dc1-05c92dd07425 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.003980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516421-8b70-4d4c-9dc1-05c92dd07425 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.012346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 68/137] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:07.014326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.014865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.015329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.016907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5f79d-4c5f-4775-a630-8d559c730197 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:07.057493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:07.057921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:07.058961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.059250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.059533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] PATH_INFO: `/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9/users/7fa620e446a2472186b5f2e19bc28fd4/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.059910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:07.062429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] RBAC: Authorizing `identity:create_grant(project_id=3a08f9f2846a43f8ae40b4be3bb5cbd9, user_id=7fa620e446a2472186b5f2e19bc28fd4, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:07.068330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:07.144056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 70/138] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:06 2022] PUT /identity/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9/users/7fa620e446a2472186b5f2e19bc28fd4/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:07.171678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 69/139] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:07.218534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:07.353335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d1385-28b7-4729-b6d2-3409c51773db None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:07.356015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:07.402029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:07.402859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.403144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.403421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.421768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:07.429620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:07.430572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:07.431334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 71/140] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:07.438249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:07.481790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:07.547944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79ba3a-b373-4ebc-997e-a3eb2709ecc7 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:07.582352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:07.583029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.583315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.583596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.584379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:07.584691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:07.586089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:07.599301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 70/141] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:07.604998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:07.715265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5e0f5-7523-46a8-8132-cf02dc66c160 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:07.751205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be9d2-f1f3-46a0-876f-7108a312a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:07.755470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be9d2-f1f3-46a0-876f-7108a312a59a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.755470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be9d2-f1f3-46a0-876f-7108a312a59a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.755470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be9d2-f1f3-46a0-876f-7108a312a59a None admin] PATH_INFO: `/v3/users/8d4e8588f0b448199c8c5e35afe88ad4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.755470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be9d2-f1f3-46a0-876f-7108a312a59a None admin] RBAC: Authorizing `identity:get_user(user_id=8d4e8588f0b448199c8c5e35afe88ad4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:07.755470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be9d2-f1f3-46a0-876f-7108a312a59a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:07.757046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 72/142] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:07 2022] GET /identity/v3/users/8d4e8588f0b448199c8c5e35afe88ad4 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:07.762514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:07.884068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f337bc-6ea7-4678-90fb-4066df4a1818 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:07.885564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:07.922296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be872e2f-1b90-4cdc-9af7-e570ff480dee None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.923267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be872e2f-1b90-4cdc-9af7-e570ff480dee None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.923782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be872e2f-1b90-4cdc-9af7-e570ff480dee None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.925427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 73/143] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:07.930585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:07.931199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.931306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:07.931515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.931798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.932183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:07.932854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:07.958224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:07.959965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:08.015743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:08.062371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:08.064211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:08.094459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 74/144] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:08.180694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:08.230669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 71/145] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:08.241789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:08.244539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:08.351374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138bc4a-b943-44cf-98ce-57e1b999210e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:08.418274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:08.419086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.419371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.419647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.424074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d15b-6878-4073-85b3-b241cfcf58c3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:08.444375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:08.454636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:08.455517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:08.456267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 75/146] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:08.464950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:08.495551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:08.496479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.496871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.497147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.498479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:08.500517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:08.502354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 72/147] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:08 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:08.508787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:08.510721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:08.623240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:08.667206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b735d-8d5a-4279-916f-bea88700bc38 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:08.673152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a7eb8-efe3-42a2-b3ee-ac57b3c8258c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:08.709034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:08.709625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.709909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.710186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.711027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:08.711334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:08.712743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e89ba9-dc89-4327-89dc-3c753bbfadf9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:08.721534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 76/148] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:08.722878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d7bf5-4bf7-48fa-893c-2cccb84df8e3 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.723326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d7bf5-4bf7-48fa-893c-2cccb84df8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.723773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d7bf5-4bf7-48fa-893c-2cccb84df8e3 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.725460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 77/149] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:08.730531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d7bf5-4bf7-48fa-893c-2cccb84df8e3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:08.766405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:08.767662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.768122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.768581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.769641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=8d4e8588f0b448199c8c5e35afe88ad4) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:08.770155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:08.771845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86547a23-f9e2-421a-96ab-2e953b72b51f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:08.786558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 73/150] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:08 2022] GET /identity/v3/role_assignments?user.id=8d4e8588f0b448199c8c5e35afe88ad4&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:08.788166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.788673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.789167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.829329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:08.830909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:08.848072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:08.879289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d7bf5-4bf7-48fa-893c-2cccb84df8e3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:08.919746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:08.920895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:08.922839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:08.923704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:08.925267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:08.926908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:08.929131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 78/151] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:08 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:08.935046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:09.062544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 74/152] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:09.076584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc8994-d19d-4cc1-afb5-77bdd6576785 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:09.133329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:09.134466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:09.134910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:09.135340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:09.156267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:09.164758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:09.235828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:09.237641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:09.297504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:09.339388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061736d-17a2-483f-8cf7-5243b3e19bca None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:09.375632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 79/153] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:09.381554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:09.386023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:09.386847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:09.387130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:09.387408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:09.393118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:09.394625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:09.396652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 75/154] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:09.401741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:09.437062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:09.487428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337cd29-b510-4724-addd-e4ae5f2df08d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:09.537063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:09.538209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:09.538633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:09.539062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:09.540495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:09.542100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:09.544981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 80/155] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:09 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:09.549865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a32422-6889-4fe7-92a3-a280ff065e22 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:09.550350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:09.585174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:09.596057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d04a7-1d37-4efc-90c3-143c7ca1c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:09.596978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d04a7-1d37-4efc-90c3-143c7ca1c012 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:09.597565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d04a7-1d37-4efc-90c3-143c7ca1c012 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:09.599128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d04a7-1d37-4efc-90c3-143c7ca1c012 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:09.600029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d04a7-1d37-4efc-90c3-143c7ca1c012 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:09.602496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d04a7-1d37-4efc-90c3-143c7ca1c012 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:09.698012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c1a5-3882-4523-8c24-0fef0922a6fa None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:09.720210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:09.733141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 76/156] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:09 2022] POST /identity/v3/users => generated 283 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:04:09.735198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:09.735977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:09.736154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:09.739284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:09.739284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=7fa620e446a2472186b5f2e19bc28fd4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:09.739284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:09.739284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3be0d2-0a70-4663-ac75-9857802d280f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:09.746673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 81/157] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:09 2022] GET /identity/v3/role_assignments?user.id=7fa620e446a2472186b5f2e19bc28fd4&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=60342ac4b2b144e98d66f95f23780594 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:10.016627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1909e-96ea-4e33-81a7-a31a48547388 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.017154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1909e-96ea-4e33-81a7-a31a48547388 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.017262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1909e-96ea-4e33-81a7-a31a48547388 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.018606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 77/158] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:10.024563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.025067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.025139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.043232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:10.044583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:10.126959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa641c-dd86-4bf2-ae48-dd43e0a9e980 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.127329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa641c-dd86-4bf2-ae48-dd43e0a9e980 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.127512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa641c-dd86-4bf2-ae48-dd43e0a9e980 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.129105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 78/159] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:10.134933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.135529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.136013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.140999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 82/160] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:10.156197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:10.157793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:10.163670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.253844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 79/161] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:10.279055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.302910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ada00-5934-41eb-8768-549c8f22af1f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:10.320473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:10.345821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:10.346614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.346895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.347171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.360771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:10.370086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:10.371000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:10.371766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 83/162] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:10.384972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.424393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f40bc-65f6-491f-a0dc-0e85693e1431 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:10.445665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:10.464279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:10.465334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.465738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.466165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.478576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:10.488385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:10.489396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:10.490173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:10.490752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 80/163] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:10.500174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.520435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42c43d-ccc5-4e43-81d4-6c2f4ae9ce5c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:10.558732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:10.559491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.559807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.560189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.560977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:10.561284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:10.562762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:10.572433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 84/164] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:10.578668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.625035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:10.632679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd652-a9c2-42a0-b7c8-6f9771153b94 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:10.676868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:10.677708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.677997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.679049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.679831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:10.680144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:10.681581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6fd5f-f379-4d96-b023-c52745554892 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:10.692158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 81/165] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:10.693459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e427527-375a-429a-8cf2-d425a6886364 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.693752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e427527-375a-429a-8cf2-d425a6886364 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.694029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e427527-375a-429a-8cf2-d425a6886364 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.695288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 82/166] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:10.698298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e427527-375a-429a-8cf2-d425a6886364 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.728002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f64c3-6f01-4053-8dc2-09795453aba5 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:10.771430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:10.789519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab7f5a-825c-4522-a0a2-00c888316288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:10.790412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab7f5a-825c-4522-a0a2-00c888316288 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.790694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab7f5a-825c-4522-a0a2-00c888316288 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.791185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab7f5a-825c-4522-a0a2-00c888316288 None admin] PATH_INFO: `/v3/users/fe8ddf9ca04e49349faf1804f40edfe7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.793222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab7f5a-825c-4522-a0a2-00c888316288 None admin] RBAC: Authorizing `identity:get_user(user_id=fe8ddf9ca04e49349faf1804f40edfe7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:10.794931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab7f5a-825c-4522-a0a2-00c888316288 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:10.798345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 85/167] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/users/fe8ddf9ca04e49349faf1804f40edfe7 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:10.799616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.799956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.800240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.821557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:10.828419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:10.859012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e427527-375a-429a-8cf2-d425a6886364 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:10.908317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 86/168] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:10.908317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.909404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:10.911833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:10.912363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:10.912849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:10.917013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:10.919310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:10.922339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 83/169] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:10.928121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:10.964085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:11.041820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6599ad0-73af-409d-91ea-e6166d88228c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.068173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ff6f7-ea0c-48a3-b209-97a9b70fd051 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.079323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:11.094495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.095378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.095719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.096003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.097288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:11.098785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:11.100531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 87/170] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:11.101301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:11.118009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.118992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.119278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.119601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.228082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:11.229608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:11.331928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 84/171] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:04:11.332711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:11.364097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5227bed-3129-4a0b-8ee0-787522dc467d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.364956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:11.376653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:11.438388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06899111-4a99-4e76-ac08-cdf9618f8d9f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.445106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.445934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.446218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.446492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.475742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:11.491494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:11.492432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:11.493297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 88/172] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:11.494197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:11.499738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.500696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.501044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.501320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/fe8ddf9ca04e49349faf1804f40edfe7/roles/9fa4a6cb6863453697d834107e908e15` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.504190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e98f0415e245278b0f5ad4f985e67c, user_id=fe8ddf9ca04e49349faf1804f40edfe7, role_id=9fa4a6cb6863453697d834107e908e15)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:11.505686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:11.601269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 85/173] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:11 2022] PUT /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c/users/fe8ddf9ca04e49349faf1804f40edfe7/roles/9fa4a6cb6863453697d834107e908e15 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:11.601961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:11.638946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd43917-9bb3-480c-a8f8-0f89077c1acb None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.710673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.711599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.711885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.712162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.714527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:11.716024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:11.717832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 89/174] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:11 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:11.724025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:11.731452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:11.790474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:11.795175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57be320-ac41-455c-b2c2-e8ca3999e866 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.865014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.865623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.865903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.866180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.867115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:11.867425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:11.869124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:11.887713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 86/175] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:11.895005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:11.928265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e5061-5474-4ad1-9905-77dbf18b2aed None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:11.993167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:11.994113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:11.994396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:11.994671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:11.995519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] RBAC: Adding query filter params (role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=7fa620e446a2472186b5f2e19bc28fd4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:11.995833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:11.999113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:12.004979 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 90/176] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:11 2022] GET /identity/v3/role_assignments?user.id=7fa620e446a2472186b5f2e19bc28fd4&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:12.057783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0d580-da80-48d6-bb72-9c5f220e4b0a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:12.100079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:12.111272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:12.118205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:12.118205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:12.118205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] PATH_INFO: `/v3/users/7fa620e446a2472186b5f2e19bc28fd4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:12.118205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa620e446a2472186b5f2e19bc28fd4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:12.119674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:12.122033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 87/177] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:11 2022] GET /identity/v3/users/7fa620e446a2472186b5f2e19bc28fd4 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:12.127737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:12.283772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:12.290884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f103d-fbd1-4b46-a9ec-9d1a1445161c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:12.328361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:12.329016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:12.329298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:12.329574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:12.401459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:12.402959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:12.499996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:12.507380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 91/178] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:12.513580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:12.674370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93e265-11e8-4026-955a-9e6deb5426b4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:12.711245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20676df7-1c4b-425e-bbfd-8ec5c82e2f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:12.712359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20676df7-1c4b-425e-bbfd-8ec5c82e2f84 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:12.712645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20676df7-1c4b-425e-bbfd-8ec5c82e2f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:12.712917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20676df7-1c4b-425e-bbfd-8ec5c82e2f84 None admin] PATH_INFO: `/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:12.714199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20676df7-1c4b-425e-bbfd-8ec5c82e2f84 None admin] RBAC: Authorizing `identity:get_project(project_id=3a08f9f2846a43f8ae40b4be3bb5cbd9)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:12.715668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20676df7-1c4b-425e-bbfd-8ec5c82e2f84 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:12.717615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 88/179] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:12 2022] GET /identity/v3/projects/3a08f9f2846a43f8ae40b4be3bb5cbd9 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:12.723920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:12.858373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac1c7-e443-447f-a64f-858f4dfa33a4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:12.902173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:12.902822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:12.903099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:12.903370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:12.904192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=3a08f9f2846a43f8ae40b4be3bb5cbd9, user.id=7fa620e446a2472186b5f2e19bc28fd4) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:12.904524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:12.906511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea584897-cb2f-4895-9b80-21c964a827c2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:12.912514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 92/180] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:12 2022] GET /identity/v3/role_assignments?user.id=7fa620e446a2472186b5f2e19bc28fd4&scope.project.id=3a08f9f2846a43f8ae40b4be3bb5cbd9&role.id=60342ac4b2b144e98d66f95f23780594 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:13.221074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c21162-d6d5-4584-a5f5-3b842197b912 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.221074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c21162-d6d5-4584-a5f5-3b842197b912 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.221074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c21162-d6d5-4584-a5f5-3b842197b912 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.221074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 89/181] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:13.227112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb442e0c-6b0c-40ee-95ea-83f14f06ec36 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.228021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb442e0c-6b0c-40ee-95ea-83f14f06ec36 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.228537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb442e0c-6b0c-40ee-95ea-83f14f06ec36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.247087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bb442e0c-6b0c-40ee-95ea-83f14f06ec36 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:13.248658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bb442e0c-6b0c-40ee-95ea-83f14f06ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:13.309156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:13.324271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 93/182] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:13.349408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c21162-d6d5-4584-a5f5-3b842197b912 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:13.434134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:13.476369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c21162-d6d5-4584-a5f5-3b842197b912 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:13.511961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecabe3-8e46-40a6-a211-87755af09c75 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.512458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecabe3-8e46-40a6-a211-87755af09c75 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.512578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecabe3-8e46-40a6-a211-87755af09c75 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.513930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 94/183] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:13.521729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874b98f-fa58-4684-8144-a7f10e8c9b1c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.522308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874b98f-fa58-4684-8144-a7f10e8c9b1c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.522929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874b98f-fa58-4684-8144-a7f10e8c9b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.526058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:13.526862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.527142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.527418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.541129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:13.544201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8874b98f-fa58-4684-8144-a7f10e8c9b1c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:13.545673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8874b98f-fa58-4684-8144-a7f10e8c9b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:13.550282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:13.551185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:13.551973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 90/184] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:13.558113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:13.631930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 95/185] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:13.640530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ca49-3ed4-4237-8b50-7a4b6a06d294 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.640530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ca49-3ed4-4237-8b50-7a4b6a06d294 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.640530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ca49-3ed4-4237-8b50-7a4b6a06d294 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.640530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 96/186] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:13.642243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.642534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.642819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.662110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:13.663302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:13.677556 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2f1a-5be0-4cfc-8f1c-c66844925f9a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:13.702192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:13.711329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:13.716778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.716778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.716778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.716778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:13.716778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:13.716778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:13.729074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 91/187] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:13.729074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:13.752426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 97/188] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:13.753844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:13.813423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:13.835388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324016b-65da-4ce8-8888-6d70c39e63c9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:13.872422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d33971-d26d-4a7b-b722-b6fcbdede49c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:13.873616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:13.875006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.875290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.875572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.886795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:13.895027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:13.895907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:13.897143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 92/189] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:13.900933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:13.911679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:13.912712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:13.913001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:13.913281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] PATH_INFO: `/v3/users/b1c899c6735f4c7695b8eb328563e632` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:13.929015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c899c6735f4c7695b8eb328563e632)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:13.930620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:13.937559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 98/190] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:13 2022] GET /identity/v3/users/b1c899c6735f4c7695b8eb328563e632 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:13.937559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:13.955332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.009537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639044ed-1c92-47c0-b3c5-cd1ddc008a41 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.044165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69566ec9-2d52-4929-bae9-e6895ad143f1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.045425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.046866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.047154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.047428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.048751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.050214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.056679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 93/191] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:14.056832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.056925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:14.082012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.083493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.083926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.084545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.086071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:14.086379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.087763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.096615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 99/192] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:14.097425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:14.169747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885dd30-aac6-4e78-871c-aaf5e590b4be None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.191706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.208804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c9cc-65f6-43b5-a88a-0d7826e1794c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.212811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.213793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.214311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.214814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.245604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.246339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.246578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.246856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.247595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.249213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.304191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.305759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.368576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 100/193] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:14 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:14.369788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:14.400332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.427104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 94/194] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:14.432868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:14.464140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.487261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc4723-0207-408c-ab66-220a1601e561 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.529098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.529959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.530239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.532586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] PATH_INFO: `/v3/users/b1c899c6735f4c7695b8eb328563e632` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.534230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] RBAC: Authorizing `identity:get_user(user_id=b1c899c6735f4c7695b8eb328563e632)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.535908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.537767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 101/195] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:14 2022] GET /identity/v3/users/b1c899c6735f4c7695b8eb328563e632 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:14.543381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:14.578067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccccab-92f5-4767-aa99-9591db90eddc None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.620323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad03a6-5ede-4324-a637-0b85732e07a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.621062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad03a6-5ede-4324-a637-0b85732e07a2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.621346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad03a6-5ede-4324-a637-0b85732e07a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.621623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad03a6-5ede-4324-a637-0b85732e07a2 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.622895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad03a6-5ede-4324-a637-0b85732e07a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.624515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad03a6-5ede-4324-a637-0b85732e07a2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.626292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 95/196] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:14 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:14.627640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca728f0-87de-4065-91f0-814a36a8cfa1 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.627844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca728f0-87de-4065-91f0-814a36a8cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.628124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca728f0-87de-4065-91f0-814a36a8cfa1 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.630591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 96/197] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:14.633304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca728f0-87de-4065-91f0-814a36a8cfa1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:14.651695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.661596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09b745-a7a5-4c3c-8545-77af3f7b540c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.702213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.703421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.703966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.704254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.745277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.763469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca728f0-87de-4065-91f0-814a36a8cfa1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:14.793175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.794842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.808474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:14.809141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.809421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.809694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.810539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=b1c899c6735f4c7695b8eb328563e632) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:14.810852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:14.812612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b717f-e4a1-4fdb-b3cf-ea0a765ca297 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:14.819645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 97/198] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:14 2022] GET /identity/v3/role_assignments?user.id=b1c899c6735f4c7695b8eb328563e632&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:14.821160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:14.821454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:14.821734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:14.856800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:14.858228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:14.912740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:14.924153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 102/199] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:04:14.930297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:15.014587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 98/200] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:15.060115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:15.092631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19394887-3242-4b22-9f87-5912d626baaf None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:15.158149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:15.160033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:15.160970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:15.162744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:15.164228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:15.166724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:15.174263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 103/201] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:14 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:15.180104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:15.193115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:15.217933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b8e59-5542-4664-acff-60828ac73b3f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:15.263083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:15.264275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:15.265814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:15.266966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:15.292871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:15.308273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:15.309738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:15.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 99/202] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:15.316630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:15.328667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:15.414158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9d392-7ed4-4796-afc9-d6eb28820037 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:15.489080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:15.489741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:15.490024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:15.490304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:15.491142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] RBAC: Adding query filter params (role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=b1c899c6735f4c7695b8eb328563e632) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:15.491455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:15.492942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:15.505111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 104/203] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:15 2022] GET /identity/v3/role_assignments?user.id=b1c899c6735f4c7695b8eb328563e632&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:15.516569 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4b0a-043c-4fc8-88d4-cc2b8dd923ec None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:15.554632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:15.555315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:15.555601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:15.555882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:15.556685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:15.556996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:15.558506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:15.564814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:15.569876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 100/204] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:15.575503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:15.713855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3f6c1-ca50-4b63-9dc4-127a792fdc5e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:15.741083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:15.752897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:15.753774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:15.754058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:15.754328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] PATH_INFO: `/v3/users/fe8ddf9ca04e49349faf1804f40edfe7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:15.755698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] RBAC: Authorizing `identity:get_user(user_id=fe8ddf9ca04e49349faf1804f40edfe7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:15.757200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:15.758852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 105/205] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:15 2022] GET /identity/v3/users/fe8ddf9ca04e49349faf1804f40edfe7 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:15.764624 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:15.916511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f9f3-6218-4e82-a103-22887b3c1d10 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:15.960971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:15.961627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:15.961909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:15.962186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:16.032544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:16.034732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:16.036361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:16.152667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 101/206] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:16.158964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:16.312777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb97a2-c5b3-4b59-bba5-bfaafb8c9606 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:16.361080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cf120-057c-4fe5-85e2-353c3650c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:16.361961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cf120-057c-4fe5-85e2-353c3650c437 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:16.362244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cf120-057c-4fe5-85e2-353c3650c437 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:16.362516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cf120-057c-4fe5-85e2-353c3650c437 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:16.363742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cf120-057c-4fe5-85e2-353c3650c437 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:16.365231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cf120-057c-4fe5-85e2-353c3650c437 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:16.366966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 106/207] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:16 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:16.373317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:16.535712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:16.538645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ef7cd-d30c-4a34-913c-719eb244183c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:16.594019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:16.594705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:16.594989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:16.595262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:16.596108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] RBAC: Adding query filter params (role.id=9fa4a6cb6863453697d834107e908e15, scope.project.id=53e98f0415e245278b0f5ad4f985e67c, user.id=fe8ddf9ca04e49349faf1804f40edfe7) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:16.596445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:16.597905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e07aac-4f92-47a1-8154-bca30e5a778b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:16.609456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 102/208] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:16 2022] GET /identity/v3/role_assignments?user.id=fe8ddf9ca04e49349faf1804f40edfe7&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=9fa4a6cb6863453697d834107e908e15 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:17.177866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e92457-c70e-4967-86a7-d8cb5a8e3b4c None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.178237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e92457-c70e-4967-86a7-d8cb5a8e3b4c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.178430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e92457-c70e-4967-86a7-d8cb5a8e3b4c None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.179935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 107/209] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:17.185401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.185695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.185971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.208435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:17.209628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:17.275398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ab1b1-e6f3-4343-8e9b-2fc4ed6cc43a None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.275714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ab1b1-e6f3-4343-8e9b-2fc4ed6cc43a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.275963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ab1b1-e6f3-4343-8e9b-2fc4ed6cc43a None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.277417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 108/210] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:17.284463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.285085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.285589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.308546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:17.308905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 103/211] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:17.310442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:17.334513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.389864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:17.407026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 109/212] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:17.434179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.437715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:17.444236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703940-9c59-494b-8fb4-b0750e2bc0a2 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:17.480087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:17.480927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.481210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.481487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.492964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:17.501050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:17.501965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:17.503178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 104/213] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:17.508309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.551866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53f46e-3fbf-4455-a5c6-b8389042b4b5 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:17.589357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:17.590651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.591127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.591691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.593089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:17.595417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:17.597359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 110/214] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:17.603110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.615083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2783e9-d296-4b01-b334-20c9343e8ab2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:17.653003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:17.653677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.653962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.654232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.654987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:17.655293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:17.657333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff1938-1e5d-46e5-80af-38336d4270e2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:17.666449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 105/215] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:17.671008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:17.671089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab99d7-fcf1-4099-aa5d-67a43add9a77 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.671089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab99d7-fcf1-4099-aa5d-67a43add9a77 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.671089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab99d7-fcf1-4099-aa5d-67a43add9a77 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.676390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 106/216] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:17.676390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab99d7-fcf1-4099-aa5d-67a43add9a77 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.692507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:17.722049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1665b3-1dfd-478c-984d-980159b4531c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:17.760145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:17.760841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.761143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.761398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.762145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:17.763603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:17.782528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab99d7-fcf1-4099-aa5d-67a43add9a77 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:17.818600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dae9b-ca1c-4f21-a420-3936b89f1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:17.819500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dae9b-ca1c-4f21-a420-3936b89f1a5f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.819784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dae9b-ca1c-4f21-a420-3936b89f1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.820063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dae9b-ca1c-4f21-a420-3936b89f1a5f None admin] PATH_INFO: `/v3/users/e340797071a048e787407d1cedde2912` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.838397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dae9b-ca1c-4f21-a420-3936b89f1a5f None admin] RBAC: Authorizing `identity:get_user(user_id=e340797071a048e787407d1cedde2912)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:17.839890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dae9b-ca1c-4f21-a420-3936b89f1a5f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:17.841694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 107/217] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:17 2022] GET /identity/v3/users/e340797071a048e787407d1cedde2912 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:17.842898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:17.843248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:17.844061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:17.860597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:17.861860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:17.903594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 111/218] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:17 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:17.912058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.940849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 108/219] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:17.970572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:17.974861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.057777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ed83-68c7-4df5-aedb-c214558fdbed None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.074839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.097090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d18fe6-fbc4-4e32-b2ce-38609751f114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.097811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d18fe6-fbc4-4e32-b2ce-38609751f114 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.098098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d18fe6-fbc4-4e32-b2ce-38609751f114 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.098376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d18fe6-fbc4-4e32-b2ce-38609751f114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.108256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3e192-2b98-4cd5-b5c4-17871077aa8e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.158969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285da5-9541-482e-af72-226daff25217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.159778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285da5-9541-482e-af72-226daff25217 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.160059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285da5-9541-482e-af72-226daff25217 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.160359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285da5-9541-482e-af72-226daff25217 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.170884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff285da5-9541-482e-af72-226daff25217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-ff285da5-9541-482e-af72-226daff25217 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:18.178635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:18.179698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:18.180577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 109/220] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:18.181524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542850-3d8e-4df0-818f-d145846a265b None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.181811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542850-3d8e-4df0-818f-d145846a265b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.182088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542850-3d8e-4df0-818f-d145846a265b None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.183777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 110/221] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:18.186593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94542850-3d8e-4df0-818f-d145846a265b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.196400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d18fe6-fbc4-4e32-b2ce-38609751f114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:18.197879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d18fe6-fbc4-4e32-b2ce-38609751f114 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:18.278311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.334874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94542850-3d8e-4df0-818f-d145846a265b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.341150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 112/222] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:18.341150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.341150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.341150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.361937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:18.363149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:18.374158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.385490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.387130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.387962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.388497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.389948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:18.390787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:18.392885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:18.404754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 111/223] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:18.406132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.438862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 113/224] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:18.444449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.547541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201871b-a471-472b-ac5a-5e4b258630e6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.573159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4cfb0-f091-4fac-98b4-854096ba7e70 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.587767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.597284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.598228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.598514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.598792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.600124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:18.601993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:18.603999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 112/225] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:18.604884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.622857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.624366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.625222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.626139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] PATH_INFO: `/v3/users/b1c899c6735f4c7695b8eb328563e632` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.626812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] RBAC: Authorizing `identity:get_user(user_id=b1c899c6735f4c7695b8eb328563e632)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:18.628933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:18.631175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 114/226] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/users/b1c899c6735f4c7695b8eb328563e632 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:18.632120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.658284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.727022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745d811-00b8-4ac9-a00f-fc2aee394d80 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.767659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.768546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151212d-0b7c-445f-99a2-2f6c77f0fead None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:18.768683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.768848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.769125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.783141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:18.794652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:18.795562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:18.796431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 113/227] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:18.802109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.810951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:18.811599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:18.813846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:18.813846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:18.813846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=e340797071a048e787407d1cedde2912) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:18.814349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:18.815959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:18.823029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 115/228] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/role_assignments?user.id=e340797071a048e787407d1cedde2912&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=60342ac4b2b144e98d66f95f23780594 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:18.824002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:18.881126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:18.962777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231245f7-596f-4cd6-8008-63d2a1b8017f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:19.004200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:19.005133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:19.005422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:19.005703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:19.006986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:19.008906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:19.011785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 114/229] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:19.017447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:19.040315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:19.082826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a550d91-ad1a-4706-bdef-76986b075f82 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:19.120860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:19.121536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:19.121822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:19.122100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:19.122847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:19.123157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:19.125167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:19.134159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 116/230] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:19.144311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:19.170009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11235151-188e-45ed-b6a0-de9aaa565a52 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:19.212324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:19.237712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:19.238633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:19.238926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:19.239314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24/users/b1c899c6735f4c7695b8eb328563e632/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:19.241383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3a39156776444cade26121f376ea24, user_id=b1c899c6735f4c7695b8eb328563e632, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:19.242856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:19.245794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947359de-a01a-4856-afce-1593e78f2ff4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:19.287330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:19.288205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:19.288513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:19.288930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] PATH_INFO: `/v3/users/b1c899c6735f4c7695b8eb328563e632` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:19.290144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c899c6735f4c7695b8eb328563e632)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:19.291599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:19.293760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 117/231] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:19 2022] GET /identity/v3/users/b1c899c6735f4c7695b8eb328563e632 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:19.299378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:19.311475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:19.347768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 115/232] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:19 2022] PUT /identity/v3/projects/4d3a39156776444cade26121f376ea24/users/b1c899c6735f4c7695b8eb328563e632/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:19.415819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6592bfe-108e-406c-96c5-2a96a2b6d542 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:19.453156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96af49f-74a6-4d80-a656-f32e03d948d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:19.454100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96af49f-74a6-4d80-a656-f32e03d948d1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:19.454384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96af49f-74a6-4d80-a656-f32e03d948d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:19.454663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96af49f-74a6-4d80-a656-f32e03d948d1 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:19.455999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96af49f-74a6-4d80-a656-f32e03d948d1 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:19.457522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96af49f-74a6-4d80-a656-f32e03d948d1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:19.459842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 118/233] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:19 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:19.469846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:19.613286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232856bf-c307-42a5-9f04-dcfdc2024642 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:19.650481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1914d-574c-46b4-a780-8a38cb3c2f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:19.651789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1914d-574c-46b4-a780-8a38cb3c2f52 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:19.652225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1914d-574c-46b4-a780-8a38cb3c2f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:19.652673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1914d-574c-46b4-a780-8a38cb3c2f52 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24/users/b1c899c6735f4c7695b8eb328563e632/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:19.655095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1914d-574c-46b4-a780-8a38cb3c2f52 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3a39156776444cade26121f376ea24, user_id=b1c899c6735f4c7695b8eb328563e632, role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:19.657281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1914d-574c-46b4-a780-8a38cb3c2f52 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:19.774580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 116/234] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:19 2022] PUT /identity/v3/projects/4d3a39156776444cade26121f376ea24/users/b1c899c6735f4c7695b8eb328563e632/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:20.578175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe433-bc37-4a7b-abe3-a966ad23896e None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:20.578664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe433-bc37-4a7b-abe3-a966ad23896e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:20.578664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe433-bc37-4a7b-abe3-a966ad23896e None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:20.580270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 119/235] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:20.585781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:20.586077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:20.586366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:20.608323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:20.610097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:20.653415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476f18-68cf-4b39-8ffe-909b93f92534 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:20.653719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476f18-68cf-4b39-8ffe-909b93f92534 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:20.653979 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476f18-68cf-4b39-8ffe-909b93f92534 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:20.655365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 120/236] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:20.660855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:20.661140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:20.661416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:20.672636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:20.680640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:20.682039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:20.731193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 117/237] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:20.764309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 121/238] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:20.773845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:20.785086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:20.788688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:20.892989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a310b5-195c-4223-b7e0-2320ce14fe8e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:20.930936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:20.931822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:20.932171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:20.932539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:20.933775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:20.935297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:20.937547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 122/239] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:20.943045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:21.036665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:21.041571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf1933-f109-42c5-81bd-bf934db583eb None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:21.065676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07326-b103-4328-84f8-b591655fc08d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:21.110923 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:21.111600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.111878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.112147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.112883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:21.114323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:21.131236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:21.132043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.132335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.132610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.150773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:21.154803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:21.156221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:21.157063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 118/240] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:21.163847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:21.197907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:21.230594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 123/241] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:20 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:04:21.304906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdb35e-2cb6-4d84-8364-12f62be3f6f6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:21.355064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:21.355774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.356170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.356479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.357259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:21.357663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:21.359156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:21.373197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 119/242] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:21.379311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:21.497951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb056a5d-5553-4389-b8c7-0ac1c888f2d1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:21.537319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:21.547352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1a082-6f16-4e2f-ae2b-1fd292513625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:21.548236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1a082-6f16-4e2f-ae2b-1fd292513625 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.552372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1a082-6f16-4e2f-ae2b-1fd292513625 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.552372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1a082-6f16-4e2f-ae2b-1fd292513625 None admin] PATH_INFO: `/v3/users/315d34dff85249c0b80524f4d8900612` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.571302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1a082-6f16-4e2f-ae2b-1fd292513625 None admin] RBAC: Authorizing `identity:get_user(user_id=315d34dff85249c0b80524f4d8900612)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:21.573184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1a082-6f16-4e2f-ae2b-1fd292513625 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:21.575623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 124/243] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:21 2022] GET /identity/v3/users/315d34dff85249c0b80524f4d8900612 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:21.582092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:21.691154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:21.698994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9ee4-7a59-4889-b336-f0749efffe7f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:21.723761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a54b-42f1-41b1-a7d6-2be8b0e013ea None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.724304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a54b-42f1-41b1-a7d6-2be8b0e013ea None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.724519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a54b-42f1-41b1-a7d6-2be8b0e013ea None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.725991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 125/244] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:21.731515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.731844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.732201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.741607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:21.742269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:21.742786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:21.743069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:21.755115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:21.755115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:21.828667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:21.830698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:21.836355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 126/245] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:21.862835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:21.956555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:21.970332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 120/246] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:21.976940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:21.998427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcb265-e780-46b7-8aa9-a0be193455b2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.021880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:22.043277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:22.044182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.044659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.044949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.056590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:22.070849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:22.071756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:22.072565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:22.073678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 127/247] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:22.080158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:22.101545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a26a24-5f26-47d3-b96d-f56989656101 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.152614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:22.153498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.153783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.154516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.156652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:22.158669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:22.161222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 121/248] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:21 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:22.167935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:22.224641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778585f0-a5bf-402b-82e9-99ccd70889ad None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.249803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:22.262216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:22.262903 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.263192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.263468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.270091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:22.270091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:22.270091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8c423-f5c1-4277-a33a-04ccfff914b4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:22.276396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 128/249] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:22.277701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390580-591f-4c70-ac4f-e549ad3a907f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.277991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390580-591f-4c70-ac4f-e549ad3a907f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.278273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390580-591f-4c70-ac4f-e549ad3a907f None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.279818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 129/250] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:22.285885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.286176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.286455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.302047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c885261-feba-4bb9-b32b-ff8b45ec9849 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.304928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:22.306148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:22.341598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:22.342753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.343038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.343878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.344182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:22.352320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] RBAC: Adding query filter params (role.id=9fa4a6cb6863453697d834107e908e15, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=315d34dff85249c0b80524f4d8900612) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:22.352320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:22.352320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:22.362719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 122/251] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:22 2022] GET /identity/v3/role_assignments?user.id=315d34dff85249c0b80524f4d8900612&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=9fa4a6cb6863453697d834107e908e15 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:22.372541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:22.446856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 130/252] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:22.482672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:22.637192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386ef52-65cb-4ce2-954c-d51f72911349 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.658296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031bad0-8bbe-4c74-b59e-d13e0eb87e86 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.693038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:22.693849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.694128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.694400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.708366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:22.723489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:22.726164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:22.727883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:22.727883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:22.727883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] PATH_INFO: `/v3/users/e340797071a048e787407d1cedde2912` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:22.730653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] RBAC: Authorizing `identity:get_user(user_id=e340797071a048e787407d1cedde2912)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:22.732160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:22.734692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 123/253] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:22 2022] GET /identity/v3/users/e340797071a048e787407d1cedde2912 => generated 297 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:22.736272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:22.737495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:22.739081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 131/254] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:22.743286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:22.745278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:22.810622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:22.946909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbb15-b3bd-44fd-b1da-6f6880ac4c53 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:22.993680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452b67-656e-41ca-9ac1-1be824266893 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:23.014246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.015271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.015939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.017156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.018290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:23.020134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:23.021833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2886faf-5670-4158-bc38-23c3389f4125 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:23.037214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 124/255] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:23.040008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31a2ea-2cee-4d3c-893d-2f68b06482df None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.042083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31a2ea-2cee-4d3c-893d-2f68b06482df None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.043474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31a2ea-2cee-4d3c-893d-2f68b06482df None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.045735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 125/256] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:23.048800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31a2ea-2cee-4d3c-893d-2f68b06482df None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:23.065957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8ee98-2f57-4d63-85c8-85594ca820c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.067159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8ee98-2f57-4d63-85c8-85594ca820c4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.067650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8ee98-2f57-4d63-85c8-85594ca820c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.068129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8ee98-2f57-4d63-85c8-85594ca820c4 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.071331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8ee98-2f57-4d63-85c8-85594ca820c4 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:23.073060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8ee98-2f57-4d63-85c8-85594ca820c4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:23.081366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 132/257] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:22 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:23.082959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.083656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.085694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.110669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:23.112209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:23.143979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:23.190291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31a2ea-2cee-4d3c-893d-2f68b06482df None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:23.226473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.227272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.227552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.227825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] PATH_INFO: `/v3/users/b1c899c6735f4c7695b8eb328563e632` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.229854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] RBAC: Authorizing `identity:get_user(user_id=b1c899c6735f4c7695b8eb328563e632)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:23.231312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:23.234787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 126/258] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:23 2022] GET /identity/v3/users/b1c899c6735f4c7695b8eb328563e632 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:23.236758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:23.274570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 133/259] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:23.276625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:23.332493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:23.436084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874133d-c495-4d78-8728-d3880d16b90f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:23.448648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad55d4-bd61-418f-b895-5f2512aeceff None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:23.486255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.486839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.487221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.487502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.501641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:23.523274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.524244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.524564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.524845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24/users/e340797071a048e787407d1cedde2912/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.526965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3a39156776444cade26121f376ea24, user_id=e340797071a048e787407d1cedde2912, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:23.532311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:23.573640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:23.575113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:23.603729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:23.629107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 127/260] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:23 2022] PUT /identity/v3/projects/4d3a39156776444cade26121f376ea24/users/e340797071a048e787407d1cedde2912/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:23.630007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:23.703941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 134/261] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:23.709450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:23.819233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4283-6343-4b96-8d46-a0102bd587b0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:23.850398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:23.860446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.861337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.861621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.861897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:23.863254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:23.866574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:23.866697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 135/262] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:23 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:23.872943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:23.935485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372315-8371-4ce6-8875-9577c6f605ed None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:23.988941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:23.990116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:23.990588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:23.991045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.002282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b75bae-9905-4795-bc1c-b3364908169c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:24.006483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:24.015061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:24.016588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:24.017371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:24.020081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 128/263] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:24.025311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:24.039642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:24.040355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.040650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.040931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.041790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=b1c899c6735f4c7695b8eb328563e632) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:24.042308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:24.043688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:24.050577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 136/264] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:23 2022] GET /identity/v3/role_assignments?user.id=b1c899c6735f4c7695b8eb328563e632&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:24.081741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:24.145650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542eebd-4a24-452e-a781-466b72293b83 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:24.184800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:24.185865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.186356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.186858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.188447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:24.188976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:24.190623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:24.201499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 129/265] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:24.207425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:24.291310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:24.353455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293534be-d6a4-4431-83c6-687b14875d0c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:24.392116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:24.393027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.393576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.393859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] PATH_INFO: `/v3/users/b1c899c6735f4c7695b8eb328563e632` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.395073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c899c6735f4c7695b8eb328563e632)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:24.396554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:24.398211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 137/266] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:24 2022] GET /identity/v3/users/b1c899c6735f4c7695b8eb328563e632 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:24.405205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:24.525928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5f27f-cfaa-4f01-8742-c18da25fa708 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:24.534401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:24.577810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440278ca-fd07-4e4d-917c-a3659757e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:24.578477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440278ca-fd07-4e4d-917c-a3659757e812 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.578762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440278ca-fd07-4e4d-917c-a3659757e812 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.579035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440278ca-fd07-4e4d-917c-a3659757e812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.658317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440278ca-fd07-4e4d-917c-a3659757e812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:24.659824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440278ca-fd07-4e4d-917c-a3659757e812 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:24.784179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 130/267] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:24.789918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:24.867128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5694cd-2bc3-4530-9b8b-2a4c6550ea8f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.867645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5694cd-2bc3-4530-9b8b-2a4c6550ea8f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.868192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5694cd-2bc3-4530-9b8b-2a4c6550ea8f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.870803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 131/268] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:24.881521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.882341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.882770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.900656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:24.902429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:24.903778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:24.918663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786506-8b3f-4f1f-9be7-ae5a91b04261 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:24.975871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:24.992261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:24.995052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:24.996353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:24.996783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:24.999010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:25.002210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:25.004929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 138/269] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:24 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:25.013467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.045646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 132/270] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:25.086508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.161779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76fc5-55d4-4551-afbe-5163709b06cc None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:25.205415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32183fcd-c36d-4f97-badc-90f999e08c6a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:25.206723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:25.207755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.208139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.208542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.210435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] RBAC: Adding query filter params (role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=b1c899c6735f4c7695b8eb328563e632) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:25.210860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:25.212453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1de6d5-4c26-4077-95e5-fce64b46dd5f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:25.228324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 139/271] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/role_assignments?user.id=b1c899c6735f4c7695b8eb328563e632&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:25.228324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b77397-94cd-4c80-89ae-e415cde53d9b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.228324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b77397-94cd-4c80-89ae-e415cde53d9b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.228324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b77397-94cd-4c80-89ae-e415cde53d9b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.228324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 140/272] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:25.232271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.232589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.232864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.255065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:25.256532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:25.281596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:25.282878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.283396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.283867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.288122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:25.290101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:25.292358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 133/273] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:25.299810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.310758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:25.324386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:25.341614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 141/274] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:25.368881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.495496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f9bb4-ae3f-4cd8-b9ae-6a929f6a6e22 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:25.537607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:25.538421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.538700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.538973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.552608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:25.563009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:25.564008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:25.564833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:25.569718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 142/275] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:25.571196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.572058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff22ee4-5bc5-4d96-a9f1-e596fa997e8b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:25.616437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefb4ba-848e-40d1-b165-dc91980c7f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:25.617766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefb4ba-848e-40d1-b165-dc91980c7f81 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.617766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefb4ba-848e-40d1-b165-dc91980c7f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.617766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefb4ba-848e-40d1-b165-dc91980c7f81 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.618339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefb4ba-848e-40d1-b165-dc91980c7f81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:25.620106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefb4ba-848e-40d1-b165-dc91980c7f81 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:25.636763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:25.689853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240effb-0dac-4ca9-bdf7-249123ec76f6 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:25.722748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:25.723617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.723897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.724172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.724967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:25.725282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:25.726793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875979f-db16-4f2d-b6e6-a7be4435145e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:25.740538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 143/276] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:25.740538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bfdb4-78a1-49e8-82e8-a946eb2a07e7 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.740538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bfdb4-78a1-49e8-82e8-a946eb2a07e7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.740538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bfdb4-78a1-49e8-82e8-a946eb2a07e7 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.740538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 144/277] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:25.741408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bfdb4-78a1-49e8-82e8-a946eb2a07e7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.760027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 134/278] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:25 2022] POST /identity/v3/users => generated 293 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:04:25.760027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.760027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.760027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.777405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:25.778951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:25.853692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:25.876244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 135/279] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:25.896241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:25.911493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bfdb4-78a1-49e8-82e8-a946eb2a07e7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:25.925291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:25.960709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:25.962276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:25.963231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:25.963977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] PATH_INFO: `/v3/users/c1385d44e7804d96ba9c5feb3ad10a54` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:25.981901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] RBAC: Authorizing `identity:get_user(user_id=c1385d44e7804d96ba9c5feb3ad10a54)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:25.983177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:25.985750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 145/280] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/users/c1385d44e7804d96ba9c5feb3ad10a54 => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:25.991271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:26.024863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246a3d1-72f2-4089-922a-cd22d2628e50 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.070619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.072155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.072478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.072760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.086529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:26.094493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:26.095416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:26.096191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 136/281] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:26.102802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:26.120235 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1cc0e-2b3a-4512-8751-5d7956e6185f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.160996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.161675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.161933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.162208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.163040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, user.id=c1385d44e7804d96ba9c5feb3ad10a54, scope.system=all) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:26.163352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:26.164932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:26.169806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:26.171125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 146/282] 10.222.0.46 () {56 vars in 1308 bytes} [Sat Jun 4 09:04:25 2022] GET /identity/v3/role_assignments?user.id=c1385d44e7804d96ba9c5feb3ad10a54&scope.system=all&role.id=60342ac4b2b144e98d66f95f23780594 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:26.235562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c3c43-26b2-4afa-b53b-7f0eeecbdc48 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.284808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.285434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.285718 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.285996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.286780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:26.287089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:26.288573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:26.302144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 137/283] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:26.309527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:26.344314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:26.436433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993d35-b28b-4491-a184-65861f705551 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.473019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b4f0-ee54-4de5-b84b-78b9cfd090ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.473899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b4f0-ee54-4de5-b84b-78b9cfd090ad None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.474180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b4f0-ee54-4de5-b84b-78b9cfd090ad None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.474450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b4f0-ee54-4de5-b84b-78b9cfd090ad None admin] PATH_INFO: `/v3/users/315d34dff85249c0b80524f4d8900612` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.475668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29b4f0-ee54-4de5-b84b-78b9cfd090ad None admin] RBAC: Authorizing `identity:get_user(user_id=315d34dff85249c0b80524f4d8900612)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:26.477147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29b4f0-ee54-4de5-b84b-78b9cfd090ad None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:26.483273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 147/284] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:26 2022] GET /identity/v3/users/315d34dff85249c0b80524f4d8900612 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:26.485350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:26.611376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd013e-be9a-45b5-b245-a95d7ff884e7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.623797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07a81b-86a3-4780-95f6-3440a8614f6f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.624106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07a81b-86a3-4780-95f6-3440a8614f6f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.624380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07a81b-86a3-4780-95f6-3440a8614f6f None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.625829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 148/285] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:26.633358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.633358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.633358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.638220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:26.651582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:26.653268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:26.677065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.678673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.678785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.679046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.685086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:26.686555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:26.688719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 138/286] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:26 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:26.694379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:26.732904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 149/287] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:26.756316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:26.812253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:26.894988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae42b-1b30-468c-aaa7-33a3501ba3e9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.944123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2c620-b273-46b9-a4fd-02f6e91edb8a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:26.954670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.955901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.956416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.956870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24/users/315d34dff85249c0b80524f4d8900612/roles/9fa4a6cb6863453697d834107e908e15` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:26.959239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3a39156776444cade26121f376ea24, user_id=315d34dff85249c0b80524f4d8900612, role_id=9fa4a6cb6863453697d834107e908e15)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:26.961032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:26.995812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:26.996971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:26.997430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:26.997931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:27.009678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:27.020109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:27.021141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:27.021956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:27.024347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 150/288] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:27.029814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:27.048621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 139/289] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:04:26 2022] PUT /identity/v3/projects/4d3a39156776444cade26121f376ea24/users/315d34dff85249c0b80524f4d8900612/roles/9fa4a6cb6863453697d834107e908e15 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:04:27.147783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:27.158338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6075f-9ce4-49b5-974e-ff033c851292 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:27.204995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:27.205680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:27.205957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:27.206228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:27.206980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:27.207286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:27.208705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:27.224173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 151/290] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:27.230438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:27.311257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:27.377363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c6627-4cbe-49f0-9864-0608d2e88662 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:27.415034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:27.415837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:27.416123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:27.416449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] PATH_INFO: `/v3/users/e340797071a048e787407d1cedde2912` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:27.417658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] RBAC: Authorizing `identity:get_user(user_id=e340797071a048e787407d1cedde2912)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:27.419125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:27.421119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 140/291] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:27 2022] GET /identity/v3/users/e340797071a048e787407d1cedde2912 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:27.429606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:27.552159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42f785-9688-45c8-b46f-6fd7a77fff00 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:27.586029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:27.586674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:27.586953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:27.587225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:27.609202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:27.674449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:27.675840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:27.789107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 152/292] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:04:27.795105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:27.924199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed74f4-2101-4a75-8fe6-6035d2fb66ed None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:27.933546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:27.958503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444100f2-6de7-4aac-8a05-7089b2288c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:27.959369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444100f2-6de7-4aac-8a05-7089b2288c82 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:27.959646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444100f2-6de7-4aac-8a05-7089b2288c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:27.959919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444100f2-6de7-4aac-8a05-7089b2288c82 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:27.961135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444100f2-6de7-4aac-8a05-7089b2288c82 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:27.962577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444100f2-6de7-4aac-8a05-7089b2288c82 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:27.964581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 141/293] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:27 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:27.970395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:28.039239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:28.103489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0221d-329d-4ff7-995c-5e3216b5c27b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:28.154372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:28.155396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:28.155936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:28.156465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:28.157499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=e340797071a048e787407d1cedde2912) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:28.158010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:28.159660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46a956-8c28-4011-b9ef-73d542947b49 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:28.168945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 153/294] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:27 2022] GET /identity/v3/role_assignments?user.id=e340797071a048e787407d1cedde2912&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=60342ac4b2b144e98d66f95f23780594 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:28.700938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adec94-b3ce-40ae-ae6d-f90caa12e71d None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:28.701468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adec94-b3ce-40ae-ae6d-f90caa12e71d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:28.701468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adec94-b3ce-40ae-ae6d-f90caa12e71d None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:28.702883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 142/295] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:28.708641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c02cf-8299-4258-88b3-f38822fccbe5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:28.709211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c02cf-8299-4258-88b3-f38822fccbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:28.709687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c02cf-8299-4258-88b3-f38822fccbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:28.729842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9f3c02cf-8299-4258-88b3-f38822fccbe5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:28.731454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9f3c02cf-8299-4258-88b3-f38822fccbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:28.813700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 154/296] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:28.838751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adec94-b3ce-40ae-ae6d-f90caa12e71d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:28.951590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adec94-b3ce-40ae-ae6d-f90caa12e71d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:28.956676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a75e2-ce88-468c-9f07-b1ef7fc95a9b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:28.956971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a75e2-ce88-468c-9f07-b1ef7fc95a9b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:28.957254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a75e2-ce88-468c-9f07-b1ef7fc95a9b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:28.958784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 155/297] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:28.963859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7a050-c6af-42e9-9f61-255e54345259 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:28.964148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7a050-c6af-42e9-9f61-255e54345259 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:28.964480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7a050-c6af-42e9-9f61-255e54345259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:28.981670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-94d7a050-c6af-42e9-9f61-255e54345259 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:28.982868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-94d7a050-c6af-42e9-9f61-255e54345259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:28.994196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5325878-0201-47ea-9339-44b2fca62664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:28.995712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5325878-0201-47ea-9339-44b2fca62664 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:28.996327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5325878-0201-47ea-9339-44b2fca62664 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:28.996678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5325878-0201-47ea-9339-44b2fca62664 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.008948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5325878-0201-47ea-9339-44b2fca62664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-a5325878-0201-47ea-9339-44b2fca62664 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:29.018358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:29.019306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:29.020105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 143/298] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:29.021599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b324-a4b1-4509-bd98-ce354292e6ec None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.021890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b324-a4b1-4509-bd98-ce354292e6ec None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.022175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b324-a4b1-4509-bd98-ce354292e6ec None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.024322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 144/299] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:29.025875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b324-a4b1-4509-bd98-ce354292e6ec None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.035896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.061998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 156/300] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:29.063113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.063395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.063671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.072217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.084467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:29.085656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:29.143215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b324-a4b1-4509-bd98-ce354292e6ec None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.154994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 157/301] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:29.155952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.188653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.189436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.189723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.189995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.190738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:29.191041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.192444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.200674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 145/302] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:29.201473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.270722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0c58d-b6fd-49b2-8163-1c90061f9948 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.281069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.310699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.313334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.314135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.314420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.314695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.315272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd6e93-4a6c-41fd-aa71-109b5fc01533 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.315864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.318083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.323834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 158/303] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:29.323834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.353006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.353865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.354149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.354418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.365761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:29.373776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:29.375398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:29.376206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 146/304] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:29.376929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.431468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b093a-27de-46e3-802a-2fc6fbd622a1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.470855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.471735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.472023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.472309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] PATH_INFO: `/v3/users/874cb229c85843bdb9abc79926ca0a79` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.485323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385920b-56c1-4cf0-8e82-0507b4f457bf None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.487363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] RBAC: Authorizing `identity:get_user(user_id=874cb229c85843bdb9abc79926ca0a79)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.488937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.495648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 159/305] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/users/874cb229c85843bdb9abc79926ca0a79 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:29.495648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.525822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.526163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.526746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.527032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.527316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] PATH_INFO: `/v3/groups` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.528022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.529506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.595384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bd95-48f4-4bf7-b96c-8c86ae486c50 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.620714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 147/306] 10.222.0.46 () {60 vars in 1134 bytes} [Sat Jun 4 09:04:29 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:29.621785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.644769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.646094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.647532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.648193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.650879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:29.651560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.653401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.657991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.665682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 160/307] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:29.673005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.761639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b5902-2a36-492c-97f4-d010ca4153fc None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.803699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.804596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.805200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.805461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.806643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] RBAC: Adding query filter params (role.id=9fa4a6cb6863453697d834107e908e15, user.id=874cb229c85843bdb9abc79926ca0a79, scope.system=all) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:29.807065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.808763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.811484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.816140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 148/308] 10.222.0.46 () {56 vars in 1308 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/role_assignments?user.id=874cb229c85843bdb9abc79926ca0a79&scope.system=all&role.id=9fa4a6cb6863453697d834107e908e15 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:29.820716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f435e-7c95-440d-8a63-0e0ddf7294aa None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:29.863054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4aed0-c953-4d73-84f6-af1db0d8a40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:29.864241 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4aed0-c953-4d73-84f6-af1db0d8a40e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:29.864700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4aed0-c953-4d73-84f6-af1db0d8a40e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:29.865140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4aed0-c953-4d73-84f6-af1db0d8a40e None admin] PATH_INFO: `/v3/users/c1385d44e7804d96ba9c5feb3ad10a54` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:29.866656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4aed0-c953-4d73-84f6-af1db0d8a40e None admin] RBAC: Authorizing `identity:get_user(user_id=c1385d44e7804d96ba9c5feb3ad10a54)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:29.868394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4aed0-c953-4d73-84f6-af1db0d8a40e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:29.870994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 161/309] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:29 2022] GET /identity/v3/users/c1385d44e7804d96ba9c5feb3ad10a54 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:29.877434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:29.975492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:29.997148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c3af-7612-4a95-914e-8c287217dd9c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:30.003385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddd47f-289c-49c6-baf1-427dc2780d7b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.003947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddd47f-289c-49c6-baf1-427dc2780d7b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.003947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddd47f-289c-49c6-baf1-427dc2780d7b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.005648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 162/310] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:30.012664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.012990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.013215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.032074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:30.033305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:30.041180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:30.042419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.042702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.042975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] PATH_INFO: `/v3/system/users/c1385d44e7804d96ba9c5feb3ad10a54/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.044528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c1385d44e7804d96ba9c5feb3ad10a54, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:30.045912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:30.111100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 163/311] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:30.134784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:30.158754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 149/312] 10.222.0.46 () {58 vars in 1334 bytes} [Sat Jun 4 09:04:29 2022] PUT /identity/v3/system/users/c1385d44e7804d96ba9c5feb3ad10a54/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:30.248391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:30.258686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bf5b9-7487-48a8-90eb-69447fb549d9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:30.305873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:30.307180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.307719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.308191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.320939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:30.330008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:30.330952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:30.331747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:30.334590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 164/313] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:30.341406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:30.451147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e878df-17a9-4aa2-a0e8-e5349c6f43cb None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:30.487148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:30.488160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.488662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.489134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.490094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:30.490607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:30.492238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:30.501312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 150/314] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:30.509427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:30.613301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b26a0-15d8-425e-9eb0-1e7d155991be None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:30.652862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:30.653739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.654022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.654302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] PATH_INFO: `/v3/users/315d34dff85249c0b80524f4d8900612` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.655525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] RBAC: Authorizing `identity:get_user(user_id=315d34dff85249c0b80524f4d8900612)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:30.657184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:30.659138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 165/315] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:30 2022] GET /identity/v3/users/315d34dff85249c0b80524f4d8900612 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:30.664313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:30.676158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:30.773522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aca61-d4dd-4ccf-91e1-1f64ca586532 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:30.807829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:30.808642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:30.808928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:30.809208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:30.919467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:30.921030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:30.932978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:31.034635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 151/316] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:31.041394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:31.066138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:31.150394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913e09-7460-42f5-a3f6-be92d5fb702d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:31.185909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6834143-178b-4cb0-81ef-68237da831e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:31.187022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6834143-178b-4cb0-81ef-68237da831e4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:31.188004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6834143-178b-4cb0-81ef-68237da831e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:31.188465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6834143-178b-4cb0-81ef-68237da831e4 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:31.189818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6834143-178b-4cb0-81ef-68237da831e4 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:31.191474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6834143-178b-4cb0-81ef-68237da831e4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:31.193411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 166/317] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:31 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:31.200320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:31.307727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2726-b50b-49b9-878a-801880a95384 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:31.317514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:31.343204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:31.343897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:31.344183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:31.344491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:31.346270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] RBAC: Adding query filter params (role.id=9fa4a6cb6863453697d834107e908e15, scope.project.id=4d3a39156776444cade26121f376ea24, user.id=315d34dff85249c0b80524f4d8900612) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:31.346616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:31.348074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f8b80-eb38-425b-817d-291ba90c65f5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:31.354668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 152/318] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:04:31 2022] GET /identity/v3/role_assignments?user.id=315d34dff85249c0b80524f4d8900612&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=9fa4a6cb6863453697d834107e908e15 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:32.106558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21674-82a5-4a66-bf7c-819ec8d316f4 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.107109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21674-82a5-4a66-bf7c-819ec8d316f4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.107109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21674-82a5-4a66-bf7c-819ec8d316f4 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.108756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 167/319] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:32.114084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.114499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.114686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.132508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:32.133716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:32.150560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c448b-8063-4015-a75f-c106108895f3 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.150848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c448b-8063-4015-a75f-c106108895f3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.151326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c448b-8063-4015-a75f-c106108895f3 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.152890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 168/320] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:32.157681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.157964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.158243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.175367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:32.176585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:32.219758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 153/321] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:32.242296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.253249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 169/322] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:32.276706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.295781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:32.362011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379b319-9b98-48a2-be3b-5f6bbaa90424 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.363967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:32.383985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a6ff4-f042-4915-8f0b-399b06762c4d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.399463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.400198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.400525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.400801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.402051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:32.403621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:32.410640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 154/323] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:32.412021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.427904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.429423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.429712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.429990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.443321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:32.451299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:32.452385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:32.453286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:32.455304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 170/324] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:32.461250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.519611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907028f-7564-4f9f-a087-8b5add890fd5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.543805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:32.557238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d624ad-8717-491c-a976-ead5fb28494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.557921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d624ad-8717-491c-a976-ead5fb28494f None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.558211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d624ad-8717-491c-a976-ead5fb28494f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.558486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d624ad-8717-491c-a976-ead5fb28494f None admin] PATH_INFO: `/v3/groups` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.559180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d624ad-8717-491c-a976-ead5fb28494f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:32.560647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d624ad-8717-491c-a976-ead5fb28494f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:32.564544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bc19b-51a1-4d04-bc87-5a48f11d18ae None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.598521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.599186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.599469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.599749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.600518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:32.600828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:32.602210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744f5c9-446f-419e-b2e2-6a087fc3ef08 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:32.610272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 171/325] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:32.611501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398746ef-14a0-4e32-bfa0-b30184d3d351 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.611792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398746ef-14a0-4e32-bfa0-b30184d3d351 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.612120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398746ef-14a0-4e32-bfa0-b30184d3d351 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.613369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 172/326] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:32.615701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398746ef-14a0-4e32-bfa0-b30184d3d351 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.643420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 155/327] 10.222.0.46 () {60 vars in 1134 bytes} [Sat Jun 4 09:04:32 2022] POST /identity/v3/groups => generated 226 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:04:32.644930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.645218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.645496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.670918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:32.672158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:32.698298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:32.734641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398746ef-14a0-4e32-bfa0-b30184d3d351 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.748707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 156/328] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:32.773088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.773610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.774156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.774438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.774769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] PATH_INFO: `/v3/users/874cb229c85843bdb9abc79926ca0a79` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.776068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] RBAC: Authorizing `identity:get_user(user_id=874cb229c85843bdb9abc79926ca0a79)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:32.777590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:32.779286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 173/329] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:32 2022] GET /identity/v3/users/874cb229c85843bdb9abc79926ca0a79 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:32.785744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.798474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:32.894792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ff8d2-2992-47e2-9acb-5c16170cad60 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.905085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728c175-5c56-4d50-9b03-36c6dea98176 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:32.935955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.937964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.938715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.938994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.945216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:32.946095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:32.946379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:32.946657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] PATH_INFO: `/v3/system/users/874cb229c85843bdb9abc79926ca0a79/roles/9fa4a6cb6863453697d834107e908e15` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:32.949019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=874cb229c85843bdb9abc79926ca0a79, role_id=9fa4a6cb6863453697d834107e908e15)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:32.950414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:32.951226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:32.960517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:32.961520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:32.962310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 157/330] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:32.967946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:32.991980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:33.021521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 174/331] 10.222.0.46 () {58 vars in 1334 bytes} [Sat Jun 4 09:04:32 2022] PUT /identity/v3/system/users/874cb229c85843bdb9abc79926ca0a79/roles/9fa4a6cb6863453697d834107e908e15 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:33.085352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c81d41-1f06-4f6c-b082-42f2e7fd7b22 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:33.127169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:33.127852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:33.128133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:33.128448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:33.129220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:33.129529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:33.130947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:33.141746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 158/332] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:33.147214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:33.227882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:33.266542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b627-be4e-409a-a0a3-f6724b691517 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:33.313601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcdf3f-e539-4eff-943c-704382b3bef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:33.314489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcdf3f-e539-4eff-943c-704382b3bef8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:33.314770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcdf3f-e539-4eff-943c-704382b3bef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:33.315046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcdf3f-e539-4eff-943c-704382b3bef8 None admin] PATH_INFO: `/v3/users/c1385d44e7804d96ba9c5feb3ad10a54` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:33.319506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcdf3f-e539-4eff-943c-704382b3bef8 None admin] RBAC: Authorizing `identity:get_user(user_id=c1385d44e7804d96ba9c5feb3ad10a54)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:33.319506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcdf3f-e539-4eff-943c-704382b3bef8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:33.319899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 175/333] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:33 2022] GET /identity/v3/users/c1385d44e7804d96ba9c5feb3ad10a54 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:33.325956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:33.433034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:33.454612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602a61-8aa7-4bfc-bb93-331e7fa9476c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:33.494958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:33.495623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:33.495906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:33.496183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:33.497047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] RBAC: Adding query filter params (role.id=60342ac4b2b144e98d66f95f23780594, user.id=c1385d44e7804d96ba9c5feb3ad10a54, scope.system=all) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:33.497418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:33.498926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd7c91-c74a-489c-a8d9-25e1b1756950 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:33.512575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 159/334] 10.222.0.46 () {56 vars in 1308 bytes} [Sat Jun 4 09:04:33 2022] GET /identity/v3/role_assignments?user.id=c1385d44e7804d96ba9c5feb3ad10a54&scope.system=all&role.id=60342ac4b2b144e98d66f95f23780594 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:35.638638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e98c9-b743-4a37-8cb5-784b697bb6a9 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:35.638638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e98c9-b743-4a37-8cb5-784b697bb6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:35.638638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e98c9-b743-4a37-8cb5-784b697bb6a9 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:35.638638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 176/335] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:35.649929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:35.650844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:35.651323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:35.670244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:35.672317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:35.694682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4eff-63bc-4e8f-bf76-fb7865e675c5 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:35.695294 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4eff-63bc-4e8f-bf76-fb7865e675c5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:35.695772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4eff-63bc-4e8f-bf76-fb7865e675c5 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:35.697373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 177/336] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:35.716323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:35.716323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:35.716323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:35.747865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:35.747865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:35.764848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 160/337] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:35.813762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:35.872317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 178/338] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:35.914402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:35.973687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfad30-5334-4d42-9c37-29b50d50846f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:35.999333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.021952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8cbda-dfd3-44de-8d44-efbaf3105f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.022994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8cbda-dfd3-44de-8d44-efbaf3105f95 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.023294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8cbda-dfd3-44de-8d44-efbaf3105f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.023572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8cbda-dfd3-44de-8d44-efbaf3105f95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.043347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.068934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8cbda-dfd3-44de-8d44-efbaf3105f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-2ff8cbda-dfd3-44de-8d44-efbaf3105f95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:36.077564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:36.078505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:36.079297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 161/339] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:36.084643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ad2b-61c0-4caa-8222-e9d89f962e7c None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.084848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b58aa-43d9-4bcd-bf42-b5b3a44b34f6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:36.084975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ad2b-61c0-4caa-8222-e9d89f962e7c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.085216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ad2b-61c0-4caa-8222-e9d89f962e7c None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.086969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 162/340] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:36.106085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.106279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.106979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.137179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.139485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.139947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.143565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.148543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:36.148543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:36.161237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:36.176048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:36.177211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:36.178251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:36.180796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 179/341] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:36.183695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:36.267399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 163/342] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:36.268308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:36.314717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96735d4d-27ad-41be-a5fc-b0f5707f195c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:36.355310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.361034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.364827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.365574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.365864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.366144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.366997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:36.367305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:36.368717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:36.378137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 180/343] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:36.378981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:36.414636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ca46d-38e2-4547-8acb-bdb00279ff83 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:36.453498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.454100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.454381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.454656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.455513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:36.455820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:36.458561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c32dc-af83-4dd7-b3c6-5e0b50bf7b94 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:36.468058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 164/344] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:36.470126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345b641-f40f-4b02-abae-d8f82f098f0f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.470416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345b641-f40f-4b02-abae-d8f82f098f0f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.470691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345b641-f40f-4b02-abae-d8f82f098f0f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.472048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 165/345] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:36.472869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345b641-f40f-4b02-abae-d8f82f098f0f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:36.494248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e1c5-7536-4fd8-a85b-bbd360e58b4e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:36.532138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862f85b-3e09-4e4e-bb51-a26ffca168f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.532987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f85b-3e09-4e4e-bb51-a26ffca168f4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.533322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f85b-3e09-4e4e-bb51-a26ffca168f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.533603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f85b-3e09-4e4e-bb51-a26ffca168f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.548199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862f85b-3e09-4e4e-bb51-a26ffca168f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9862f85b-3e09-4e4e-bb51-a26ffca168f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:36.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:36.559737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:36.561738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:36.564521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 181/346] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:36.568306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.568306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.568306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.584176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:36.585044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345b641-f40f-4b02-abae-d8f82f098f0f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:36.585533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:36.621531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.631612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.632567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.635031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.635031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.656461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.682992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 182/347] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:36.684776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:36.724534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:36.726046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:36.820853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82974f-2c84-4beb-81d8-ff3f3570af8f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:36.849227 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 166/348] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:36.850001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:36.866481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:36.867457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:36.867755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:36.868034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:36.899876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.934628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:36.960419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:36.961966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:36.972887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390e7e-777b-4d1e-8179-dd86d153a989 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.016080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.016756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.017047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.017326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.018093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:37.018446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.019939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.029389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 167/349] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:37.030216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.100442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 183/350] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:37.101624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.160895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:37.177623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9d38-e831-4855-ae77-dea2d1c2bc6a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.208712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:37.219534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.220648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.220937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.221217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.231987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e98cd5-e3f5-48e2-9d0d-419dadaafbb9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.236779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:37.246617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:37.247553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:37.248392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 168/351] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:37.249553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.282170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.283151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.283440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.283718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.286095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.287563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.290014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 184/352] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:37.290857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.378809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5e1bd-fafc-4ce5-9aa5-c1a264ec7c97 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.425742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8558a-bf7d-44c2-accf-06cf49f7c808 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.428579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.429245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.429530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.429807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.447147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:37.477700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.478725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.479105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.479484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.480852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.483096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.485525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 185/353] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:37.486960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.510939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:37.531800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.531800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.618611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a1614-ab93-41a8-a2b4-7a9b1815f726 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.663636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 169/354] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:37.664880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.673048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.673728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.674001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.674275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.675021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:37.675323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.678726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d54e0-a905-459f-97a7-864e516b68d5 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.688605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 186/355] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:37.690054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a6997-6e9b-4023-a01d-2b454e35cc47 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.690347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a6997-6e9b-4023-a01d-2b454e35cc47 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.690628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a6997-6e9b-4023-a01d-2b454e35cc47 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.691984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 187/356] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:37.692595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a6997-6e9b-4023-a01d-2b454e35cc47 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.728027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:37.795161 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e00c07-08ea-4e5e-8fd5-d7c80eeb5c33 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.804922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:37.827534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a6997-6e9b-4023-a01d-2b454e35cc47 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.840802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.841343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.841882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.841882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.850641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.852118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.854606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 170/357] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:37.855230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.872976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15999cd-3323-434d-bc0e-18a7f69087d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:37.873850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15999cd-3323-434d-bc0e-18a7f69087d2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.874135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15999cd-3323-434d-bc0e-18a7f69087d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.874414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15999cd-3323-434d-bc0e-18a7f69087d2 None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.875659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15999cd-3323-434d-bc0e-18a7f69087d2 None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:37.877194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15999cd-3323-434d-bc0e-18a7f69087d2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:37.878857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 188/358] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:37.880033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:37.880978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:37.881351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:37.899658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:37.900899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:37.974968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 189/359] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:37.975927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:37.984951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9044-4115-4037-a2e3-c797c6283174 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:37.994381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:38.029910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.031065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.031916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.032194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.033773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.034939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.036939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 171/360] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.037591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.065345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:38.090874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451386f-bc31-4dad-ad50-79aa8b2beb0d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.133133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127e36e-11f9-4480-bbb1-8b39d18ea951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.134305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127e36e-11f9-4480-bbb1-8b39d18ea951 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.134587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127e36e-11f9-4480-bbb1-8b39d18ea951 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.134865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127e36e-11f9-4480-bbb1-8b39d18ea951 None admin] PATH_INFO: `/v3/users/874cb229c85843bdb9abc79926ca0a79` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.136197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127e36e-11f9-4480-bbb1-8b39d18ea951 None admin] RBAC: Authorizing `identity:get_user(user_id=874cb229c85843bdb9abc79926ca0a79)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.137759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127e36e-11f9-4480-bbb1-8b39d18ea951 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.139582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 190/361] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:04:37 2022] GET /identity/v3/users/874cb229c85843bdb9abc79926ca0a79 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.140811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a981bc-5e16-4779-af1f-bcce573c82b2 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.141595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a981bc-5e16-4779-af1f-bcce573c82b2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.142143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a981bc-5e16-4779-af1f-bcce573c82b2 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.143280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 191/362] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:38.143861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a981bc-5e16-4779-af1f-bcce573c82b2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.169990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178ebf6-568d-4801-8ef2-8e970d8b29e7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.216165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.216592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.216873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.217148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.217981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=53e98f0415e245278b0f5ad4f985e67c) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:38.218289 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.219778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.227297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 172/363] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.228371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.276622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a981bc-5e16-4779-af1f-bcce573c82b2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.298622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:38.319696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.320331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.320605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.320929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.321782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=53e98f0415e245278b0f5ad4f985e67c) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:38.322103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.323602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.331787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 192/364] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=60342ac4b2b144e98d66f95f23780594 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.332621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.354472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:38.389802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff6280-d0b7-4a92-a649-49416843cc7a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.445437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.446619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.447092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.447546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.467543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:38.479288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:38.480265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:38.481096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:38.482717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 173/365] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:38.482717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.487529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5662-ac1b-4167-8186-273fd8b17a12 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.531315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e8874-6954-41b0-8feb-01b0855ebb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.532629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e8874-6954-41b0-8feb-01b0855ebb85 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.533121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e8874-6954-41b0-8feb-01b0855ebb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.533607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e8874-6954-41b0-8feb-01b0855ebb85 None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.535148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e8874-6954-41b0-8feb-01b0855ebb85 None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.536973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e8874-6954-41b0-8feb-01b0855ebb85 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.539416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 193/366] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.540954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.541558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.542035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.566027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:38.567568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:38.628767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:38.635591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776068b-8176-4df8-9bb0-64009fea56cb None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.654269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 194/367] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:38.655765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.667554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:38.689035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.689907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.690018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.690253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.691109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] RBAC: Adding query filter params (role.id=9fa4a6cb6863453697d834107e908e15, user.id=874cb229c85843bdb9abc79926ca0a79, scope.system=all) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:38.691407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.692974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.716335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 174/368] 10.222.0.46 () {56 vars in 1308 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/role_assignments?user.id=874cb229c85843bdb9abc79926ca0a79&scope.system=all&role.id=9fa4a6cb6863453697d834107e908e15 => generated 510 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.716335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.781588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ce42b-842c-4cc3-b8a1-871561377cd5 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.828688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.829736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.830332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.830840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.831877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:38.832492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.834112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.844322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9ed17-03f4-4882-887e-765f1fa4c271 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:38.847030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 195/369] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.849161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.901288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:38.901982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:38.902264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:38.902543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:38.903422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=4d3a39156776444cade26121f376ea24) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:38.903717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:38.909601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:38.920206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 175/370] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=60342ac4b2b144e98d66f95f23780594 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:38.921150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:38.985395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:39.008668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:39.042594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd65fd-5155-4d83-b01d-e62ebbabf844 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:39.115764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:39.116775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:39.117141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:39.117610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:39.137112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:39.149335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:39.150352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:39.151220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 196/371] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:39.164198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:39.200663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b706d8-7b12-4690-85f8-4ea2fa0302c2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:39.265089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:39.265801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:39.266087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:39.266363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:39.347354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c641300-6735-4aae-b83c-312f7adc4281 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:39.357762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:39.359745 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:39.383894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:39.385926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:39.392963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:39.393649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:39.393934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:39.394213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:39.394973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:39.395282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:39.396759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:39.406580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 197/372] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:39.412229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:39.469280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 176/373] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:39.476585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:39.582910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9f44-9122-4711-b25a-2d4fd43a7e14 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:39.630250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d725c90-9efe-4044-a519-fe35a963d428 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:39.658689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:39.659357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:39.659643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:39.660223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:39.660404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:39.673842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:39.674747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:39.675042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:39.675768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:39.679577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:39.681079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:39.682846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 177/374] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:39 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:39.688975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:39.789057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:39.806801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:39.808300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:39.895885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5dae-f228-4749-b56c-1372aa16b916 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:39.926871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 198/375] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:39.939999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:39.945086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:39.945977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:39.946265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:39.946544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:39.948200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:39.950732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:39.952446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 178/376] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:39 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:39.958684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:40.044760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:40.053842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c277c57-2273-43f8-8191-b045e4f8dd59 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:40.094631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:40.095583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:40.095919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:40.096199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] PATH_INFO: `/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:40.098070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd4724a5a83452ab3b4b7b3823eccc9)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:40.099705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:40.102023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 199/377] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:39 2022] GET /identity/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:40.103956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:40.113341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:40.187271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520ae2-8b30-4174-a2ad-cdfb7f45c87c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:40.279655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:40.280472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:40.280747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:40.281063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:40.284007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=4d3a39156776444cade26121f376ea24) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:40.284409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:40.285838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:40.291960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc708c-731c-406a-8162-52ef40673512 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:40.302620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 179/378] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:39 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 260 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:40.338874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a542cb2-6d1b-4581-b687-5dde81a0b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:40.339833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542cb2-6d1b-4581-b687-5dde81a0b546 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:40.340100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542cb2-6d1b-4581-b687-5dde81a0b546 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:40.340397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542cb2-6d1b-4581-b687-5dde81a0b546 None admin] PATH_INFO: `/v3/groups/98e9f3d57cb04b22b0069901dec3a40c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:40.349444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a542cb2-6d1b-4581-b687-5dde81a0b546 None admin] RBAC: Authorizing `identity:get_group(group_id=98e9f3d57cb04b22b0069901dec3a40c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:40.350994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a542cb2-6d1b-4581-b687-5dde81a0b546 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:40.353098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 200/379] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:40 2022] GET /identity/v3/groups/98e9f3d57cb04b22b0069901dec3a40c => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:40.359457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:40.480073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a371456-e99c-494e-886b-dc87232cabf3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:40.517359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:40.533312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:40.535878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:40.535878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:40.535878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:40.535878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] RBAC: Adding query filter params (group.id=98e9f3d57cb04b22b0069901dec3a40c, role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=fbd4724a5a83452ab3b4b7b3823eccc9) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:40.535878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:40.537169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25200a48-4d79-4f6f-bc20-dce771bb4baa None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:40.549363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 180/380] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:40 2022] GET /identity/v3/role_assignments?group.id=98e9f3d57cb04b22b0069901dec3a40c&scope.project.id=fbd4724a5a83452ab3b4b7b3823eccc9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:41.373540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33cdb6-e6d6-4e61-bb46-9485cfa090e6 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:41.374050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33cdb6-e6d6-4e61-bb46-9485cfa090e6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:41.374127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33cdb6-e6d6-4e61-bb46-9485cfa090e6 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:41.375809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 201/381] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:41.381085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:41.381491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:41.381658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:41.409352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:41.410588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:41.464879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55afd-9dfe-4997-8af3-43b8ba105a7e None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:41.465204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55afd-9dfe-4997-8af3-43b8ba105a7e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:41.465440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55afd-9dfe-4997-8af3-43b8ba105a7e None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:41.467130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 202/382] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:41.480005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:41.480322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:41.480589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:41.494196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 181/383] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:41.505267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:41.506479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:41.538234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:41.559826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:41.599704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 203/384] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:41.640415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:41.770366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd1337-2a30-46db-a839-aebb44cbc822 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:41.798043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08f43b-b744-450d-b9b0-fe8c374a85cd None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:41.816168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:41.868250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:41.869157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:41.870080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:41.870360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:41.872737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:41.873739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:41.874021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:41.874296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:41.899507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:41.911254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:41.912205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:41.913628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:41.914238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:41.914310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 204/385] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:41.920012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:41.921858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:41.923975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:41.925472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:41.925630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 182/386] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:41.933390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:41.954208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:42.091422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95afd0-5687-4041-81f1-e75838674081 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.134644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.135682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.136172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.136871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.137854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:42.138382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.139977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.151240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 183/387] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:42.157503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.189411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76dfe9-1675-4579-b4c1-4f00b5916af1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.230675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:42.263729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.268819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.272699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.273236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.275318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:42.275841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.278157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcf755-0c8f-41ed-891d-f9275a1ceaab None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.303758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 205/388] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:42.303758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621208b-7dea-451a-a99a-8b7618d5f6c6 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.303758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621208b-7dea-451a-a99a-8b7618d5f6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.303758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621208b-7dea-451a-a99a-8b7618d5f6c6 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.305082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 206/389] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:42.308484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621208b-7dea-451a-a99a-8b7618d5f6c6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.314850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fba6a-8ef7-4910-9351-9747f1c09915 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.351084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d16fc-71df-43ed-b18a-80807dc5f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.352350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d16fc-71df-43ed-b18a-80807dc5f55a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.352848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d16fc-71df-43ed-b18a-80807dc5f55a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.353326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d16fc-71df-43ed-b18a-80807dc5f55a None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.354791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d16fc-71df-43ed-b18a-80807dc5f55a None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.356514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d16fc-71df-43ed-b18a-80807dc5f55a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.358732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 184/390] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:42 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:42.360864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.361361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.361861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.380411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:42.381923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:42.403880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:42.430943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621208b-7dea-451a-a99a-8b7618d5f6c6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.456619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 185/391] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:42.458855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.471696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.473068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.473572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.474048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.475476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.477151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.479552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 207/392] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:42 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:42.485245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.521406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:42.591964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6d6fb-a595-4dab-a2c7-c3256606f416 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.613061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e0db6-988f-4075-8c36-30939c5d9311 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.632111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.633303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.633732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.634180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.635577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.637252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.644845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 186/393] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:42 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:42.646713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.653656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.654825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.655315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.655791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.657912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.659610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.661630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 208/394] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:42 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:42.662741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.690041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:42.778356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c6ec-f2b4-47e4-bded-9713bdee550a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.796682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:42.826468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b5d62-7fb1-4feb-9b2e-8719a0792575 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:42.827909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.829277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.830719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.831083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.847572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:42.860903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:42.861832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 187/395] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:42.863127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:42.875683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:42.876621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:42.876909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:42.877189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c/groups/510d95b71afa491f80941d4a0582acfa/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:42.879832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] RBAC: Authorizing `identity:create_grant(project_id=53e98f0415e245278b0f5ad4f985e67c, group_id=510d95b71afa491f80941d4a0582acfa, role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:42.882123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:42.988318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 209/396] 10.222.0.46 () {58 vars in 1442 bytes} [Sat Jun 4 09:04:42 2022] PUT /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c/groups/510d95b71afa491f80941d4a0582acfa/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:42.988318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.005156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790fa2a-ff44-4b2c-ab4f-177d0bb83cf8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.082968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:43.101063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b515d-4eb5-407a-b035-f7c9c0261a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.102398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b515d-4eb5-407a-b035-f7c9c0261a2b None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.102890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b515d-4eb5-407a-b035-f7c9c0261a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.103386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b515d-4eb5-407a-b035-f7c9c0261a2b None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c/groups/510d95b71afa491f80941d4a0582acfa/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.111193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b515d-4eb5-407a-b035-f7c9c0261a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=53e98f0415e245278b0f5ad4f985e67c, group_id=510d95b71afa491f80941d4a0582acfa, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:43.113049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b515d-4eb5-407a-b035-f7c9c0261a2b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:43.126352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bec70-a0c4-49cb-a3b4-39ee5332f08b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.178699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.179666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.179977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.180260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.181068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:43.181401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:43.183118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2df3e1-6725-478e-9dc4-7c845ab97d24 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:43.193830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 210/397] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:43.197354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa02b-972a-414e-9ad0-5d34ee23c307 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.202667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa02b-972a-414e-9ad0-5d34ee23c307 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.202667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa02b-972a-414e-9ad0-5d34ee23c307 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.202667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 211/398] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:43.205570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5aa02b-972a-414e-9ad0-5d34ee23c307 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.206527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 188/399] 10.222.0.46 () {58 vars in 1442 bytes} [Sat Jun 4 09:04:42 2022] PUT /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c/groups/510d95b71afa491f80941d4a0582acfa/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:43.210522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de22e2-3962-4b14-b2fa-62dbc95d2e2c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.210884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de22e2-3962-4b14-b2fa-62dbc95d2e2c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.212401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de22e2-3962-4b14-b2fa-62dbc95d2e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.257591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-31de22e2-3962-4b14-b2fa-62dbc95d2e2c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:43.258821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-31de22e2-3962-4b14-b2fa-62dbc95d2e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:43.295074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:43.352774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5aa02b-972a-414e-9ad0-5d34ee23c307 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.365489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 189/400] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:43.367777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a07c7-fd1b-4c9b-972d-7842d12b4894 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.368060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a07c7-fd1b-4c9b-972d-7842d12b4894 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.368362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a07c7-fd1b-4c9b-972d-7842d12b4894 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.371474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 190/401] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:43.377581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.378117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.378401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.402191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:43.403748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:43.405561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.408581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.409184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.409665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.411409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:43.413228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:43.416425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 212/402] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:43 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:43.418147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.422141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:43.503006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 191/403] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:43.503789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.590661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625e3e-4236-4ea2-9807-ce90141e5d09 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.623343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:43.637142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.637142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.637142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.637142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.649287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:43.649473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887efe34-c5ae-4f34-951d-6e0b4d5fa72a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:43.660097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:43.661086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:43.661959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 213/404] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:43.662609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.697247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.698931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.699731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.700213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.701942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:43.705128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:43.707603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 192/405] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:43 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:43.708735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.746652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:43.793303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c456b-c259-470d-9611-224f11255118 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.831797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.832189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.832512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.832793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.844222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:43.846711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b39146-3d53-4b44-b4ca-dad09213b12f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:43.852124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:43.853471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 214/406] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:43.854347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.890822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:43.891842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:43.892419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:43.892897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:43.893947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:43.894456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:43.896235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:43.911354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 193/407] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:43.912711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:43.930926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:43.963636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d707-25c7-497e-8a2e-af3221b997ab None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:43.988684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:44.001968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.003231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.003380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.003534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24/groups/510d95b71afa491f80941d4a0582acfa/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.005916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3a39156776444cade26121f376ea24, group_id=510d95b71afa491f80941d4a0582acfa, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.007959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.029172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51789792-32e6-4bd2-a20c-6c469f173026 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.066190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.066869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.067157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.067426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.068177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:44.068512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.069893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.079918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 194/408] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:44.080539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:44.106418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 215/409] 10.222.0.46 () {58 vars in 1442 bytes} [Sat Jun 4 09:04:43 2022] PUT /identity/v3/projects/4d3a39156776444cade26121f376ea24/groups/510d95b71afa491f80941d4a0582acfa/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:04:44.109072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:44.267799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18d32a-5ecd-4ee1-94d6-398194a6c775 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.279617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:44.331984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.333649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.334555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.335020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] PATH_INFO: `/v3/groups/98e9f3d57cb04b22b0069901dec3a40c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.336602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] RBAC: Authorizing `identity:get_group(group_id=98e9f3d57cb04b22b0069901dec3a40c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.339187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.340111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671475-501b-4ecc-9983-4c46efcf9a8b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.341288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 216/410] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:44 2022] GET /identity/v3/groups/98e9f3d57cb04b22b0069901dec3a40c => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:44.347649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:44.383870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.385098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.385753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.386194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.387589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.389266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.391616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 195/411] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:44 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:44.397567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:44.431483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:44.522321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fb1b9-1895-4b48-9bea-14323280a9da None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.554815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7a7e-08b5-4a69-bb86-35fad25619cd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.577260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.579045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.579535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.580104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] PATH_INFO: `/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.584446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd4724a5a83452ab3b4b7b3823eccc9)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.586356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.589074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 217/412] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:44 2022] GET /identity/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9 => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:44.595950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:44.596759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.597689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.597975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.598252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.599564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.603103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.603444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 196/413] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:44 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:44.610506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:44.628705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:44.799086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a7442-fb8e-4a20-9e34-f77c6b981b13 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.821719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455badd-a597-4438-b85f-f60a3fae0575 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:44.859598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:44.871949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb16d-2fd5-44a2-a7a3-4063dc1ded2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.878668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb16d-2fd5-44a2-a7a3-4063dc1ded2b None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.878668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb16d-2fd5-44a2-a7a3-4063dc1ded2b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.878668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb16d-2fd5-44a2-a7a3-4063dc1ded2b None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24/groups/510d95b71afa491f80941d4a0582acfa/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.878668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb16d-2fd5-44a2-a7a3-4063dc1ded2b None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3a39156776444cade26121f376ea24, group_id=510d95b71afa491f80941d4a0582acfa, role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.879097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb16d-2fd5-44a2-a7a3-4063dc1ded2b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.885566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d7937-2b4b-4c0b-bd45-7d2ba9af1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:44.886452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d7937-2b4b-4c0b-bd45-7d2ba9af1980 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:44.886730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d7937-2b4b-4c0b-bd45-7d2ba9af1980 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:44.887005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d7937-2b4b-4c0b-bd45-7d2ba9af1980 None admin] PATH_INFO: `/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9/groups/98e9f3d57cb04b22b0069901dec3a40c/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:44.889763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d7937-2b4b-4c0b-bd45-7d2ba9af1980 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd4724a5a83452ab3b4b7b3823eccc9, group_id=98e9f3d57cb04b22b0069901dec3a40c, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:44.891972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d7937-2b4b-4c0b-bd45-7d2ba9af1980 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:44.957682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 197/414] 10.222.0.46 () {58 vars in 1442 bytes} [Sat Jun 4 09:04:44 2022] PUT /identity/v3/projects/4d3a39156776444cade26121f376ea24/groups/510d95b71afa491f80941d4a0582acfa/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:04:44.993462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 218/415] 10.222.0.46 () {58 vars in 1442 bytes} [Sat Jun 4 09:04:44 2022] PUT /identity/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9/groups/98e9f3d57cb04b22b0069901dec3a40c/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:04:45.673344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11663826-89d2-4c2f-b35b-7672e0389ffe None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:45.674366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11663826-89d2-4c2f-b35b-7672e0389ffe None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:45.674862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11663826-89d2-4c2f-b35b-7672e0389ffe None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:45.676707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 198/416] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:45.682248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ed533-a629-44ba-9f90-5e4a9e1f3978 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:45.683061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ed533-a629-44ba-9f90-5e4a9e1f3978 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:45.683563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ed533-a629-44ba-9f90-5e4a9e1f3978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:45.709283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0c7ed533-a629-44ba-9f90-5e4a9e1f3978 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:45.710817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0c7ed533-a629-44ba-9f90-5e4a9e1f3978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:45.795856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 219/417] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:45.832214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11663826-89d2-4c2f-b35b-7672e0389ffe None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:45.931711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:45.970506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11663826-89d2-4c2f-b35b-7672e0389ffe None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:46.014519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:46.015347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.016156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.016477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.038173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:46.043077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eec52-f179-4c48-9caf-9201f8e37e3e None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.043376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eec52-f179-4c48-9caf-9201f8e37e3e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.043663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eec52-f179-4c48-9caf-9201f8e37e3e None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.045319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 220/418] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:46.048156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:46.049233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:46.050237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:46.050983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.050983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.051177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.052937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 199/419] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:46.061313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:46.096441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:46.097653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:46.193342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb0475-d387-4022-9823-aa42d7a52fed None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:46.229515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:46.236109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:46.236850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.237136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.237417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.240708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:46.240708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:46.240708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:46.249471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 200/420] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:46.255654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:46.270491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 221/421] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:46.312397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:46.373745 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63187655-37c6-4558-8ba3-e8f5e1118a21 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:46.412898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:46.413916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.414346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.414559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:46.414776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.481691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:46.490388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:46.492486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:46.567927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2102f7-2cb3-4b02-a377-cc8b55419411 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:46.618197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:46.619006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.619286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.619558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.654056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 201/422] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:46.659715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:46.660182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:46.681936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:46.683359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:46.684150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:46.685372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 222/423] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:46.694040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:46.832744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:46.841234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6ba0-38a0-4b07-a48f-4e38ec33f915 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:46.845463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:46.853728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d1b4-fc0c-45e5-be0e-dc3df47f0b4a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:46.907551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:46.908445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.908729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.909001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.910600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:46.912530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:46.915462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:46.916139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:46.920591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:46.920908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:46.921700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] RBAC: Adding query filter params (name=member) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:46.922539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:46.923948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:46.925716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 202/424] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:46.928062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:46.936676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 223/425] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:46.943622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.082291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c371204-2281-4267-b5f4-452b58854b36 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.083700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdefa9-4aba-4c4e-b11a-51145822e71e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.124930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906db9-b530-4073-9796-937a2e7f2394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.126738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906db9-b530-4073-9796-937a2e7f2394 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.127025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906db9-b530-4073-9796-937a2e7f2394 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.127301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906db9-b530-4073-9796-937a2e7f2394 None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.128640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44906db9-b530-4073-9796-937a2e7f2394 None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:47.129925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.130202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44906db9-b530-4073-9796-937a2e7f2394 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:47.130582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.130868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.131147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.132542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 203/426] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:47.133816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f9c1c-84e4-437f-af59-2796ccce1a4f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.134096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f9c1c-84e4-437f-af59-2796ccce1a4f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.134371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f9c1c-84e4-437f-af59-2796ccce1a4f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.135803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 204/427] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:47.140755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f9c1c-84e4-437f-af59-2796ccce1a4f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.167530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:47.181683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:47.216526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:47.218346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:47.282007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f9c1c-84e4-437f-af59-2796ccce1a4f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.334765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.335359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.335642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.335920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.336807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=53e98f0415e245278b0f5ad4f985e67c) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:47.337119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:47.338565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b88a9-9d72-4752-b8d4-c0a2cced933f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:47.352322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 205/428] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:47.352322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.352322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.352322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.361211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 224/429] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:04:47.369142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.385299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:47.389693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:47.503735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:47.525974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 206/430] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:47.570327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fe9bf-035f-4f3c-9a55-f275ac7eaf47 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.601451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.618823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.620517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.620948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.621367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] PATH_INFO: `/v3/projects/53e98f0415e245278b0f5ad4f985e67c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.622956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] RBAC: Authorizing `identity:get_project(project_id=53e98f0415e245278b0f5ad4f985e67c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:47.625403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:47.628396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 225/431] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/projects/53e98f0415e245278b0f5ad4f985e67c => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:47.634494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.637666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:47.740391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a756e-b60d-4e25-9f53-4c2ab8da7e0c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.759001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcccee-eb7f-4224-805f-04a0e5edbd03 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.790533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35562fb3-80bc-4c0e-b07b-53c66ca2dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.791647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35562fb3-80bc-4c0e-b07b-53c66ca2dd80 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.791931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35562fb3-80bc-4c0e-b07b-53c66ca2dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.792204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35562fb3-80bc-4c0e-b07b-53c66ca2dd80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.796705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:47.805041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35562fb3-80bc-4c0e-b07b-53c66ca2dd80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:47.805352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbd658-a2f6-40d2-9c06-13d0b4ffe9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.806209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbd658-a2f6-40d2-9c06-13d0b4ffe9eb None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.806493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbd658-a2f6-40d2-9c06-13d0b4ffe9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.806771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbd658-a2f6-40d2-9c06-13d0b4ffe9eb None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.807957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbd658-a2f6-40d2-9c06-13d0b4ffe9eb None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:47.809431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbd658-a2f6-40d2-9c06-13d0b4ffe9eb None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:47.811227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 226/432] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:47.818927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266734d4-33fa-4b63-85b7-6d4d6456fb67 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.818927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266734d4-33fa-4b63-85b7-6d4d6456fb67 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.818927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266734d4-33fa-4b63-85b7-6d4d6456fb67 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-35562fb3-80bc-4c0e-b07b-53c66ca2dd80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:47.819245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:47.820269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:47.821324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 207/433] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:47.822531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 227/434] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:47.822621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.822621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.822621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.826579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266734d4-33fa-4b63-85b7-6d4d6456fb67 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.840391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:47.841598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:47.907597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 208/435] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:47.908474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:47.923689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:47.943286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266734d4-33fa-4b63-85b7-6d4d6456fb67 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:47.977643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:47.978629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.978907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.979182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.980008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=53e98f0415e245278b0f5ad4f985e67c) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:47.980421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:47.981901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992742b-929f-4fe8-b87d-266df439ec8b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:47.988342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 228/436] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=53e98f0415e245278b0f5ad4f985e67c&role.id=60342ac4b2b144e98d66f95f23780594 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:47.989580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814f8cc-aed6-4f58-9458-cd47a125725a None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:47.989870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814f8cc-aed6-4f58-9458-cd47a125725a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:47.990146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814f8cc-aed6-4f58-9458-cd47a125725a None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:47.991395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 229/437] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:47.992115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814f8cc-aed6-4f58-9458-cd47a125725a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.010851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a72e2c-7cc7-4930-b36b-a473dd54cbf6 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.051128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.051787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.052069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.052374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.053152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:48.053459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.054912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3885d-be95-44ef-841b-5bd7447b4fc2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:48.064607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 209/438] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:48.065755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.066048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.066327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.069836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.083684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:48.084946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:48.121897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814f8cc-aed6-4f58-9458-cd47a125725a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.168430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.168430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.168430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.168761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 210/439] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:48.168861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.168934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.184329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.194881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:48.202747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:48.203688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:48.204523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 230/440] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:04:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:48.205174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.302190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828a17-8ba2-4881-aa67-7bd4ea7db369 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.343195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407400-b32d-4611-8b17-460ba3e83419 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.347568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.348674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.349153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.349567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.370438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.386031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.386974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.387332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.387619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.400717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:48.410361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:48.411516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:48.412446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:48.416472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 231/441] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:48.417361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.442882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.444396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:48.454557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.548222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a855efd-52a6-4564-a9a2-26af5c7c706d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.581245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 211/442] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:48.582121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.593356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.594626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.594910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.595192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.596348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:48.596664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.598392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:48.607212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 232/443] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:48.608117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.655717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.690405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec078d94-9e98-4e1c-a2f3-6fae5967127d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.698961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.716586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d2e9b-8b31-46f3-8e40-c52b99ed8cc2 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.726364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.727033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.727312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.727776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.728555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:48.728864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.730245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:48.739197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 212/444] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:48.740063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.755857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.756777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.759797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.759797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.759797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.760498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:48.761672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 233/445] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:48.762449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:48.845258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed8c84-6e22-43f8-a84b-96c3a8aedd2b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.864205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44293351-9fe6-44a8-a5e3-3c3f2fa252e8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:48.874797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.878985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.879639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.879920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.880198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.899824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:48.900448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:48.902929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:48.902929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:48.915604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:48.949380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.950682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:48.975041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:48.976533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.049212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 213/446] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:49.050064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.073395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 234/447] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:49.074237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.085561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:49.115622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:49.150838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e1436-71dc-4130-966f-53ee6861a2bc None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.171651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b3207-149c-4c10-8643-50d4db3b8053 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.186972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.188420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.188701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.188972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.190149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.191583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.193552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 214/448] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:49.194366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.207157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.207944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.208232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.208544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] PATH_INFO: `/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.209750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd4724a5a83452ab3b4b7b3823eccc9)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.211241 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.213189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 235/449] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/projects/fbd4724a5a83452ab3b4b7b3823eccc9 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:49.213759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.301826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d57d5-5390-4fe3-872b-162e884e571c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.302588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:49.325140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3989a-6687-4c65-9ba1-caaa95ebed87 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.336366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.337238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.337520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.337799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] PATH_INFO: `/v3/projects/4d3a39156776444cade26121f376ea24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.339057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3a39156776444cade26121f376ea24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.340463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.342173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 215/450] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/projects/4d3a39156776444cade26121f376ea24 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:49.342940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.347119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:49.364949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.365613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.365892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.366165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.367002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=60342ac4b2b144e98d66f95f23780594, scope.project.id=4d3a39156776444cade26121f376ea24) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:49.367316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.368794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.375753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 236/451] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=60342ac4b2b144e98d66f95f23780594 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:49.376613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.476750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f28dd-3488-41c5-a41d-bcb4f41b5356 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.515770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f7ec-2599-4996-95b0-9e857f1a4c91 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.518813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.519991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.520265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.520581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] PATH_INFO: `/v3/groups/98e9f3d57cb04b22b0069901dec3a40c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.521928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] RBAC: Authorizing `identity:get_group(group_id=98e9f3d57cb04b22b0069901dec3a40c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.523414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.530487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 216/452] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/groups/98e9f3d57cb04b22b0069901dec3a40c => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:49.531341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.562461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.563363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.563641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.563916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] PATH_INFO: `/v3/groups/510d95b71afa491f80941d4a0582acfa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.565563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] RBAC: Authorizing `identity:get_group(group_id=510d95b71afa491f80941d4a0582acfa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.566705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.573398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 237/453] 10.222.0.46 () {56 vars in 1180 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/groups/510d95b71afa491f80941d4a0582acfa => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:49.574625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:49.591408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:49.624029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:49.663941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ff1cb-54fc-44cb-8ffd-987cb4c516d8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.705949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.706607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.706890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.709087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.709201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76099fef-d700-4b51-a7e6-2448589d151d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:49.709280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] RBAC: Adding query filter params (group.id=98e9f3d57cb04b22b0069901dec3a40c, role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=fbd4724a5a83452ab3b4b7b3823eccc9) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:49.709280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.709770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb888f-c682-4df5-beea-b366fccf53f6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.718505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 217/454] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/role_assignments?group.id=98e9f3d57cb04b22b0069901dec3a40c&scope.project.id=fbd4724a5a83452ab3b4b7b3823eccc9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:49.760745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:49.760745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:49.760745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:49.760745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:49.761576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] RBAC: Adding query filter params (group.id=510d95b71afa491f80941d4a0582acfa, role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=4d3a39156776444cade26121f376ea24) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:49.761576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:49.761576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb41ce-89ec-4a00-9405-685aca1f055c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:49.766623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 238/455] 10.222.0.46 () {56 vars in 1376 bytes} [Sat Jun 4 09:04:49 2022] GET /identity/v3/role_assignments?group.id=510d95b71afa491f80941d4a0582acfa&scope.project.id=4d3a39156776444cade26121f376ea24&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:52.640738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f90d98-94d4-4a93-b9e4-da2ebe907f36 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:52.641826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f90d98-94d4-4a93-b9e4-da2ebe907f36 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:52.641826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f90d98-94d4-4a93-b9e4-da2ebe907f36 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:52.643146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 218/456] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:52.648864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357ee2b-0eec-45b4-9577-8e6f458046fd None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:52.649332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357ee2b-0eec-45b4-9577-8e6f458046fd None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:52.649407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357ee2b-0eec-45b4-9577-8e6f458046fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:52.670829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4357ee2b-0eec-45b4-9577-8e6f458046fd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:52.672087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4357ee2b-0eec-45b4-9577-8e6f458046fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:52.766691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 239/457] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:52.799181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f90d98-94d4-4a93-b9e4-da2ebe907f36 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:52.926049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c19d-0e9e-4259-9e83-1ddd83849120 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:52.926736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c19d-0e9e-4259-9e83-1ddd83849120 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:52.927237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c19d-0e9e-4259-9e83-1ddd83849120 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:52.929246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 240/458] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:52.934776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:52.935374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:52.935884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:52.953043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:52.966518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:52.967710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:52.971177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f90d98-94d4-4a93-b9e4-da2ebe907f36 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.025738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.026533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.026808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.027077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.031415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:53.049179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.050640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:53.070214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:53.071135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:53.072004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:53.072004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:53.072004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.072004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:53.074046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 219/459] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:53.082578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.117684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 241/460] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:53.148150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.247572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa319f-984b-4790-bc00-771e0ccadbb5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.286610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.287217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08d174-f608-4ffe-8027-b9ac9e2c6eab None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.287677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.288096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.288640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.289630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:53.290079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.291590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e8c5-e145-47bc-b87a-3868c7af0c9f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.305353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 220/461] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:53.307949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7799fea-2fbb-4638-9822-a120c30df220 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.308881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7799fea-2fbb-4638-9822-a120c30df220 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.309582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7799fea-2fbb-4638-9822-a120c30df220 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.313215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 221/462] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:53.315476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d520c-cebf-41ad-b6e4-386c4d9d0f60 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.316202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d520c-cebf-41ad-b6e4-386c4d9d0f60 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.316927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d520c-cebf-41ad-b6e4-386c4d9d0f60 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.319584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 222/463] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:53.320838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d520c-cebf-41ad-b6e4-386c4d9d0f60 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.332682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.333540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.333740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.334176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.336428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:53.345835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.347287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-3dc31cbe-05c8-4793-8211-c8be08d78506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:53.364646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:53.365561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 242/464] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccbab8-f01b-4cdf-969c-6ceae7ceba85 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccbab8-f01b-4cdf-969c-6ceae7ceba85 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.369707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccbab8-f01b-4cdf-969c-6ceae7ceba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.382551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:53.387304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d6ccbab8-f01b-4cdf-969c-6ceae7ceba85 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:53.388499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d6ccbab8-f01b-4cdf-969c-6ceae7ceba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:53.439716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d520c-cebf-41ad-b6e4-386c4d9d0f60 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.459715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 243/465] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:53.461307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.461784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.462239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.478947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:53.480412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:53.487480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.488402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.488837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.489826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.490717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.493219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.559426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 244/466] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:53.559426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.600262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:53.604091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 223/467] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:53 2022] POST /identity/v3/users => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:53.605298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.669292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edd3fa-33ad-4e8c-aee5-59253efe6218 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.719252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.720369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.720858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.721358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.722300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:53.722862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.724657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.739351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 245/468] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:53.740715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.765749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c6c3-940f-43ac-91d8-5b93d6a69020 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.769077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:53.817750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.818483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.818767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.819042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.831777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.833413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:53.855270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:53.856463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:53.857505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:53.857505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:53.857505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.857505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:53.857505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 224/469] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:53.857505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:53.873580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799cd6-3080-4510-b8ac-83990f6ba8cb None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:53.884358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:53.919015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:53.919842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:53.920132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:53.920456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:53.932725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:53.934260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:53.954790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:53.955691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:53.956560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:53.956560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:53.956560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:53.956560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:53.956560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 246/470] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:53.957342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:54.000930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ac146-7bb8-458e-a600-bfff78099a72 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:54.038120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:54.038802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.039094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.039373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.040248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:54.042282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:54.078696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489df07-6818-480b-9f35-b01d5fb2ad05 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:54.114660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:54.115385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.115664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.115936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.116683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:54.116988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:54.118352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:54.135328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 247/471] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:54.135328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:54.150619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 225/472] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:53 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:54.150619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:54.168016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:54.191377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:54.309067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f975c9f-172b-484a-b815-c3468c953476 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:54.354903 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:54.355251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.355427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.355705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.356590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:54.356897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:54.358289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9fa3c-bbdc-4594-a148-d84e974e362b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:54.371716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadbdd-2fc8-42b9-837f-ee69dfc68d86 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:54.372410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 248/473] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:54.373810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92caba-8855-4e9f-a3f2-cd2e25926320 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.374098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92caba-8855-4e9f-a3f2-cd2e25926320 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.374377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92caba-8855-4e9f-a3f2-cd2e25926320 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.380201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 249/474] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:54.381020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92caba-8855-4e9f-a3f2-cd2e25926320 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:54.410739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9141525-d230-47e0-a83d-423129c32e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:54.411419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141525-d230-47e0-a83d-423129c32e92 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.411727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141525-d230-47e0-a83d-423129c32e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.412005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141525-d230-47e0-a83d-423129c32e92 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.413029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9141525-d230-47e0-a83d-423129c32e92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:54.414198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9141525-d230-47e0-a83d-423129c32e92 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:54.480344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:54.504272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92caba-8855-4e9f-a3f2-cd2e25926320 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:54.520544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 226/475] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:54 2022] POST /identity/v3/users => generated 248 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:54.521631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.521950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.522259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.556710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:54.557902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.558389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.558886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.559947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:54.561781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:54.565532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:54.566777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:54.647634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 250/476] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:04:54 2022] POST /identity/v3/users => generated 249 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:04:54.666573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:54.673423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 227/477] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:54.712513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:54.839443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d30ef-4eba-4652-b84f-55413124c001 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:54.877587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:54.879624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:54.879624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:54.879624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:54.879624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:54.880964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:54.883242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:54.929936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.sql.core [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 04 09:04:54.929936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 04 09:04:54.929936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [parameters: {'id': '747426a3489d45a8b56bae3300c40a6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 04 09:04:54.929936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80589) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application result = self._query(query) Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application conn.query(q) Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application result.read() Jun 04 09:04:54.948947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return ref.to_dict() Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 04 09:04:54.949863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application next(self.gen) Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application yield resource Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application next(self.gen) Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self.session.rollback() Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self.force_reraise() Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise self.value Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application session.commit() Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 04 09:04:54.950773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._prepare_impl() Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self.session.flush() Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._flush(objects) Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application compat.raise_( Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application flush_context.execute() Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rec.execute(self) Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 04 09:04:54.951630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 04 09:04:54.954860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application result = self._query(query) Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application conn.query(q) Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application result.read() Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 04 09:04:54.956632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [parameters: {'id': '747426a3489d45a8b56bae3300c40a6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:54.958337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 04 09:04:54.959964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:54.961955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 251/478] 10.222.0.46 () {60 vars in 1131 bytes} [Sat Jun 4 09:04:54 2022] POST /identity/v3/roles => generated 144 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:54.962064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:55.081262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4eed1c-a782-4af2-b8b3-22aaae7aa032 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:55.119766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65df9c-4cb5-49e0-a86b-d118bcf11a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:55.121198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65df9c-4cb5-49e0-a86b-d118bcf11a38 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:55.121660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65df9c-4cb5-49e0-a86b-d118bcf11a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:55.122276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65df9c-4cb5-49e0-a86b-d118bcf11a38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:55.133887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65df9c-4cb5-49e0-a86b-d118bcf11a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-1a65df9c-4cb5-49e0-a86b-d118bcf11a38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:55.142967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:55.143835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 04 09:04:55.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:55.147013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 228/479] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:04:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:55.156317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:55.289959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4f20-2e1a-4c7d-a671-68dc54bef92e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:55.326355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:55.327606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:55.328205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:55.329338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:55.330837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:55.331478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:55.334212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813782c5-4ab0-4e21-bccc-c6f598d6fba1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:55.345328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 252/480] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:56.320137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f0df-cd35-48fb-a7b7-a7b179ca9e35 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:56.320715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f0df-cd35-48fb-a7b7-a7b179ca9e35 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:56.320715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f0df-cd35-48fb-a7b7-a7b179ca9e35 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:56.322419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 229/481] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:56.327918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b632c-f856-412e-baa4-a893880446e0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:56.328431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b632c-f856-412e-baa4-a893880446e0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:56.328641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b632c-f856-412e-baa4-a893880446e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:56.345954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-739b632c-f856-412e-baa4-a893880446e0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:56.347141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-739b632c-f856-412e-baa4-a893880446e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:56.426608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:56.437557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 253/482] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:56.476076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f0df-cd35-48fb-a7b7-a7b179ca9e35 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:56.564972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:56.625581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f0df-cd35-48fb-a7b7-a7b179ca9e35 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:56.673203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38cd44-9772-4071-8ecf-6751c3169817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:56.674007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38cd44-9772-4071-8ecf-6751c3169817 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:56.674285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38cd44-9772-4071-8ecf-6751c3169817 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:56.674556 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38cd44-9772-4071-8ecf-6751c3169817 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:56.697296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38cd44-9772-4071-8ecf-6751c3169817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-0b38cd44-9772-4071-8ecf-6751c3169817 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:56.705666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:56.706628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:56.707428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 230/483] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:56.716159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b632c-f856-412e-baa4-a893880446e0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:56.837786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c37e6-1ad2-4eb9-b001-861d64ebee65 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:56.838211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c37e6-1ad2-4eb9-b001-861d64ebee65 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:56.838402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c37e6-1ad2-4eb9-b001-861d64ebee65 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:56.841123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 231/484] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:04:56.846005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:56.846291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:56.846567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:56.863702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:56.865312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:56.882839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b632c-f856-412e-baa4-a893880446e0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:56.945437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:56.946117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:56.946397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:56.946672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:56.947414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:56.947715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:56.949132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:56.963245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 254/485] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:04:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:56.964553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.006910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 232/486] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:04:57.036400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:57.057226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.098938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc48a0-76eb-4218-bd85-4db75751188e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.144781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:57.182045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.182866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.183150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.183433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.203114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b9230-bbd7-47d7-af1e-e42a6e67d8a0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.217236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:57.218721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:57.243178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.244408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.244877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.245593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.259560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:57.267798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:57.268714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:57.269538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:57.269538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:57.269538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:57.269538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:57.275648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:57.276740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:57.281964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:57.282624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 255/487] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:57.284123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 233/488] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:04:57.286705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.296202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.410047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f845182-e601-4d09-8c8f-d3dbc436eac4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.444689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:57.460347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.460347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.460347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.460347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.464009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:57.464009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:57.464009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:57.482908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 234/489] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:04:57.490150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.492270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c67302-b1b4-4efa-a7cc-bf7fd24cfdfb None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.494812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:57.566991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.567654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.567935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.568674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.568962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:57.569272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:57.570648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a62589-8226-42f2-b43d-44d9a96c5ff0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:57.588469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 256/490] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:57.588469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075a194-6a29-4910-9e94-5121cb7a9ff3 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.588469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075a194-6a29-4910-9e94-5121cb7a9ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.588469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075a194-6a29-4910-9e94-5121cb7a9ff3 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.591747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 257/491] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:57.593032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963414a-4ca1-40ab-be5c-b3a459db45e6 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.593318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963414a-4ca1-40ab-be5c-b3a459db45e6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.593595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963414a-4ca1-40ab-be5c-b3a459db45e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.611506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4963414a-4ca1-40ab-be5c-b3a459db45e6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:57.612748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4963414a-4ca1-40ab-be5c-b3a459db45e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:57.613202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965f597-e4c0-49b2-b9e8-95243e80471e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.654919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.656188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.657270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.657746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.668720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:57.670488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:57.691270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:57.692185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:57.693092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:57.693092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:57.693092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:57.693092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:57.693780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 235/492] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:57.698552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.726588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:57.814296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd554d11-f4b2-4f11-9ca4-92bf503e68e0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.817555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 258/493] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:57.818909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb08b4-51b1-43c4-a8dc-a7c8c2ce0f51 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.819191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb08b4-51b1-43c4-a8dc-a7c8c2ce0f51 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.819466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb08b4-51b1-43c4-a8dc-a7c8c2ce0f51 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.824860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 259/494] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:57.824860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb08b4-51b1-43c4-a8dc-a7c8c2ce0f51 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:57.869022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.869823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.870102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.870644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.879113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:57.880630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-1b999aa3-8292-4dbb-ba06-85142bd02069 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:04:57.897278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:57.898374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:57.899467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 236/495] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:57.904170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.904484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.904771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.923686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:57.924839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:57.948919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:57.957068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb08b4-51b1-43c4-a8dc-a7c8c2ce0f51 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:57.994049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:57.994748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:57.995030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:57.995609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:57.996382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:57.996851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:57.998339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:57.999403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 237/496] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:58.000179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.015643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.027006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 260/497] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:58.027006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.126166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e9f54-31e6-45bd-a7d0-6dd871a27b70 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.143376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaa450-41f8-43ff-ad59-35b67c787eab None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.168277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.169199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.169552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.169903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.181199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.181595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:58.181789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.182071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.182346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.183177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:58.183485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.184950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:58.190012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:58.191169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:58.192182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:58.198252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 238/498] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:04:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:58.198252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.231850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 261/499] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:58.231850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.245099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.286246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.318891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a8604-b466-44e8-903b-e702e24fdd0c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.345634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580857f-1ea5-4875-bd4a-6c5fbbfff45f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.361797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.362621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.362900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.363178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.375832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:04:58.384137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:58.385868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:04:58.386691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:58.388658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 239/500] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:58.389644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.394736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.395957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.396488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.396968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.405810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.408221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:04:58.427320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:58.428391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:04:58.429276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:58.429778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 262/501] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:58.430578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.504489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b391-37cb-4240-9912-8de702ae1f0c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.507841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.535032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.537919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b137846-7d89-400d-b127-39bf910f8424 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.540846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.541014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.541304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.541584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.542338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:58.542649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.544048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.553637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 240/502] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:58.553637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.573600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.574251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.574559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.574837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.647962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.649477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.662517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594788b-cae3-463a-a689-f0aa207da25d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.699094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.699762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.700045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.700351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.701098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:58.701403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.702791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c45b40-53aa-4cd4-94cf-e55d9d05d10a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.711142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 241/503] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:58.712673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdb401-2579-48b9-ab84-fa83a0d87e4c None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.712989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdb401-2579-48b9-ab84-fa83a0d87e4c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.713225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdb401-2579-48b9-ab84-fa83a0d87e4c None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.714585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 242/504] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:04:58.715160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdb401-2579-48b9-ab84-fa83a0d87e4c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.723253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.754616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 263/505] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:58.755390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.776881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:58.823021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdb401-2579-48b9-ab84-fa83a0d87e4c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.857741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.858084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8cac-22d9-441e-b8d7-2d3354a1eeca None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:58.858411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.858695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.858976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.860023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:58.860380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.861818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.892345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:58.893140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.893424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.893701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.898377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 243/506] 10.222.0.46 () {56 vars in 1137 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:58.899217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:58.907509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:58.909035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-92d08752-78d8-4a22-91dc-32b6012fb73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:58.926514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:58.927500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:58.928590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:58.928590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:58.928590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:58.928590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:58.928590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 264/507] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:58.928914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:58.929198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:58.929475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:58.945468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:04:58.946700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:04:58.965857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.007922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777eea0-9069-4111-a8e6-9c8f1c5a988d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.020478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 265/508] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:04:59.021350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.034120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.055545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45351495-9640-4962-b895-ab093829c30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.056380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45351495-9640-4962-b895-ab093829c30e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.056668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45351495-9640-4962-b895-ab093829c30e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.056945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45351495-9640-4962-b895-ab093829c30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.067789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45351495-9640-4962-b895-ab093829c30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.069714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45351495-9640-4962-b895-ab093829c30e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-45351495-9640-4962-b895-ab093829c30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:59.088059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:59.089157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:59.095470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:59.095470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:59.095470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.095470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:59.095470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 244/509] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:59.095470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45351495-9640-4962-b895-ab093829c30e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.125867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de87a73-1cb8-4de2-8e63-a3888ca011e3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.160197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.161037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.161317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.161595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.172060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.173665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:59.193236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:59.194267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:59.199766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:59.199766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:59.199766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.199766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:59.200030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45351495-9640-4962-b895-ab093829c30e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.200123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 266/510] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:59.200123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.206742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.236188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.236890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.237180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.237457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.273512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.301517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc17ffe-b3c6-4f63-85be-cf0704a9e807 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.309403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.310903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.335931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.336615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.336899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.337176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.337892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:59.338195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.339563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.351343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 267/511] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:59.352175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.417991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.421990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 245/512] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:04:59.423342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.454724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee64c60-50b1-470d-a62f-667c00dc3d90 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.488595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.489774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.490564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.490841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.491114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.502029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.503477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:59.521273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:59.522875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:59.523737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:59.523737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:59.523737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.523737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:59.523737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 268/513] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:59.523737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.524056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667631c-d45d-450f-8f7c-a49807fcade8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.563789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.564513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.564797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.565090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.565909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:59.566243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.567651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.581270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 246/514] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:59.582236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.648505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aca48-de8c-4f12-bada-01e023c3d37e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.666746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.686562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.687224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.687509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.687785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.688544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:59.688853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.690223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.699622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f538c2-8b08-42af-a938-df9ee77a233f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.701693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 269/515] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:59.702569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.727088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.735446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.736242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.736563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.736845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.744028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.745838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:04:59.760811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:59.764970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 247/516] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:59.765913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.809287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc616e4-afd8-43f6-836e-36a73af6c8bc None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.843098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.843882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.844775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.845063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.855178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.856791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.864763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910fe46-29d3-4c1b-9b9c-6d126a45875e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:04:59.874549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:04:59.875965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:04:59.877148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:04:59.877148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:04:59.877148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:04:59.877148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:04:59.877148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 270/517] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:04:59.877480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.902100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:04:59.902768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:04:59.903054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:04:59.903346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:04:59.904067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:04:59.904421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:04:59.905813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:04:59.909159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.917592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 248/518] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:04:59.918399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:04:59.966856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:04:59.977652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0e92d-d8fa-4a62-99e2-b8cc69fdcd22 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.011609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.012424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.012703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.012970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.020373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.020707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68184063-060f-4ee8-92a5-723d28824c0c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.021807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:00.036182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:00.037312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 271/519] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:00.038355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.058951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.059809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.060036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.060337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.067246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.068760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:00.083480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:00.084415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:00.085264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 249/520] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:00.085757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.137906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.147248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2609-80ae-4e28-b2e3-7b4f1c3a923c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.181814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.182482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.182769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.183046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.183880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:00.184195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.185676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.186930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fd445-343a-4b12-9e2c-06d54b5fb28d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.194278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.202672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 272/521] 10.222.0.46 () {56 vars in 1137 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:00.209338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.221051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.221714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.222018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.222275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.223002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:00.223308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.224723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.237457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 250/522] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:00.238282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.304599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89871-053a-430f-98a9-c922b62a583b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.338309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.338900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.339179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.339458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.340151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.341613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.345609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23474d3-9499-4f06-bc8e-18f855c5500a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.378528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.379200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.379489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.379823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.380637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:00.381017 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.382460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.399754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 251/523] 10.222.0.46 () {56 vars in 1139 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:00.400587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.424813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.452610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 273/524] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:05:00 2022] POST /identity/v3/users => generated 247 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:00.459701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.459701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.510681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809900d-c13d-4a47-aceb-b0a5bda73ee6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.555301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.556020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.556366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.556558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.557354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:00.559279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.559279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.581328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 252/525] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:00.581694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45691975-c706-4a0c-a98e-35372becbe46 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.584124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.636809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.637397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.637681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.637964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.696238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.713652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8b071-016a-4652-899e-0803534e9bd1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.719264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.720774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.723909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.755985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.756834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.757143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.757393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.764642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.770004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:00.781652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:00.782512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:00.783352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 253/526] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:00.783814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.839842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 274/527] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:00.839842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:00.900794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa93fcb-43df-45d1-92c7-26130d54c5e4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.935812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.936486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.936772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.937052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.945312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0515-b9c7-4a06-97f9-03331b2292e7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:00.947870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:00.986018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:00.986694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:00.986980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:00.987258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:00.988114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=5c1d511f4c6341e09d54bf8709efdf8e) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:00.988467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:00.989972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.008334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 275/528] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/role_assignments?user.id=5c1d511f4c6341e09d54bf8709efdf8e&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:01.008334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.080791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:01.128143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464c8a0-37ae-4aef-80ec-6598bf454ac7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.157036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.158463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.179702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:01.180751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:01.181043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:01.182813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:01.183634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:01.183950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.185635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.206089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 276/529] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:01.211843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.242348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:01.290799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 254/530] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:01.292472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.337614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d766aee-a87d-409c-80ec-9f1509ecc0c8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.355144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:01.382162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:01.383434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:01.383720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:01.383994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:01.396651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.398757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:01.423902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:01.424835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:01.425688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:01.425688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:01.425688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:01.425688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:01.430960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 277/531] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:01.430960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.442038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8c474-2917-4e1e-a986-e94cc621f5e4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.485576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:01.486308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:01.486592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:01.486876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:01.487985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=e53e659f7faf403b9f33f393d166dda3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:01.488316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.490275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.498010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 255/532] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/role_assignments?user.id=e53e659f7faf403b9f33f393d166dda3&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:01.499146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.521115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:01.570116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cbe40-bd82-4dbc-9d51-020a788bcb5c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.628495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:01.629324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:01.629615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:01.629895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:01.665575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.667180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.702731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c075f1-6def-4224-8439-bd9d4ffc91b5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:01.718032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:01.718948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:01.720371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:01.720371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:01.720371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:01.720371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:01.720371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 278/533] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:01.721664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:01.728502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.748210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:01.748210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:01.748210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:01.748210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:01.749008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:01.750430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.752027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.765716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 256/534] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:01.774576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:01.881974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a2b78-320c-454e-8575-ed441d36221c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.940648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:01.949274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:01.949960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:01.950242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:01.950519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:01.951243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:01.951549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:01.953005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:01.966184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a6c44-ae65-4fae-be16-1a2b582a1b43 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:01.979996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 279/535] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:01.986748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:02.030758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:02.032665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.033170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.033664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.050531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:02.052350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:02.082714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:02.083613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:02.084470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:02.087140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 257/536] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:02.093726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:02.116104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639ea45-4b9c-40fd-9c1e-f49f797fa5c4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:02.153608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:02.177366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:02.178547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.181011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.181547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.197820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:02.199621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:02.225370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:02.226256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:02.227089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:02.236819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 280/537] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:02.242031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:02.262470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050365c7-02ed-488a-be13-223ed1f44ac0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:02.316960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:02.332700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:02.333380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.333666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.337608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.337608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:02.337608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:02.337608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:02.360258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 258/538] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:02.367094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:02.371779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f946-eff6-4e8c-8e91-bba65200c37c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:02.419163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:02.419832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.420114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.420415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.421204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:02.421517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:02.422992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:02.449378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 281/539] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:02.461309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:02.517631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f868199-5664-430c-ba10-a58a04a5a565 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:02.544778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:02.568137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:02.568848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.569128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.569408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.570240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=8b3a9e1e2e3d438f92286a4158c8d4cd) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:02.570550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:02.574353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab854b-8b96-40e5-8101-443582b58a1a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:02.584712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 259/540] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:02 2022] GET /identity/v3/role_assignments?user.id=8b3a9e1e2e3d438f92286a4158c8d4cd&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:02.694192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f95ed-4ac5-4486-b3bf-6cc698b99249 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:02.716996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:02.748890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:02.749566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.749849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.750127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.751542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=e7837f3811b04542a6e7f56e7fa118d3) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:02.751856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:02.753354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d24a7-b1cd-431c-baa2-de4a766c7912 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:02.765610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 282/541] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:02 2022] GET /identity/v3/role_assignments?user.id=e7837f3811b04542a6e7f56e7fa118d3&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:02.855513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b41df0-f261-467a-a166-fed8e7f57c8c None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.856006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b41df0-f261-467a-a166-fed8e7f57c8c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.856006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b41df0-f261-467a-a166-fed8e7f57c8c None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.857747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 260/542] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:02.863977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:02.864268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:02.864569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:02.886335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:02.887584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:02.991119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 283/543] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:03.032260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b41df0-f261-467a-a166-fed8e7f57c8c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:03.158608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b41df0-f261-467a-a166-fed8e7f57c8c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:03.206985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:03.207799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:03.208081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:03.208389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:03.223604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:03.231626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:03.232618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:03.233728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 261/544] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:03.239379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:03.407683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc8c35-e9e3-4a48-bb92-4922d57ec356 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:03.420894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:03.457570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:03.458254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:03.458535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:03.458811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:03.459560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:03.459866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:03.461272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24807f2c-591b-4f5d-810a-8693009bf039 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:03.476939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 284/545] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:03.482159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:03.505010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:03.626708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05280db-0561-442d-b982-f796b56a6be7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:03.662945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:03.663756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:03.664037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:03.664333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:03.675253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:03.676800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:03.686474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c63dd-6f74-4aa0-b945-9049f70ff279 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:03.686768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c63dd-6f74-4aa0-b945-9049f70ff279 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:03.687072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c63dd-6f74-4aa0-b945-9049f70ff279 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:03.689221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 285/546] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:03.694274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:03.694839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:03.694839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:03.695935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:03.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:03.698604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:03.698604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:03.698604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:03.698604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:03.698604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 262/547] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:03.702581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:03.729830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:03.731024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:03.784300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:03.859302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d36a1-417e-46f2-9323-1cb5488f5573 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:03.869524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 286/548] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:03.901479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:03.931840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:03.932815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:03.933343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:03.933769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:03.934661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:03.938949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:03.940689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:03.963950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 263/549] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:03.965070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.073983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:04.080158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45619-3d9f-409a-8a28-21b94075611b None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.119932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d99db-933f-484a-8b3c-4b6a10bcbfaf None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.147062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:04.154511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.155057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.155344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.155620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.166500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.167611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.168047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.168564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.183306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:04.185319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:04.186052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:04.195244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:04.196394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:04.197405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:04.213658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:04.214686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:04.215595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:04.216110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 287/550] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:04.218193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 264/551] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:04.222724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.223787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.360932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c48a4-2fca-4092-b7aa-b6aba7cd3b6e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.374680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f37b0a-63e7-49e1-963a-86c7a52922ea None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.419953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.421865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.422480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.424049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.425148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:04.425682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:04.427914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:04.440568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.441463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.441889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.442309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.443309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:04.445751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:04.447350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:04.447760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 288/552] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:04.451550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.480966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 265/553] 10.222.0.46 () {56 vars in 1135 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:04.486513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.489581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:04.561800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:04.609184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42a095-096a-443f-aa98-313a3863fd69 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.632906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790422-fb86-44fc-b8c7-627d0ab5ec00 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.653172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.654406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.654697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.655269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.669059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:04.670558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:04.680240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.681208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.681694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.682172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-483d66e9-2ab0-41ed-b3c3-4bc34a37fc48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:04.689951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:04.690877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:04.691849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:04.691849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:04.691849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:04.691849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:04.691849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 289/554] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:04.692911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5adf2-e494-43e4-8cb7-3f8ed209df71 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.693194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5adf2-e494-43e4-8cb7-3f8ed209df71 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.693471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5adf2-e494-43e4-8cb7-3f8ed209df71 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.695517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 290/555] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:04.696888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5adf2-e494-43e4-8cb7-3f8ed209df71 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.761732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:04.764829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:04.776113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:04.809590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:04.813909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5adf2-e494-43e4-8cb7-3f8ed209df71 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:04.852010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:04.852660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.852997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.853275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.854217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:04.854328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:04.855658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023df2b-d68e-44b3-b0ab-b1969085ceb6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:04.869460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 291/556] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:04.871136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:04.871426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:04.871705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:04.881590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 266/557] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:05:04.883226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.889828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:04.891073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:04.967865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 292/558] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:04.968690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:04.994613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03cde0-9a15-4e64-9d7e-e619c10c3ad4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.035314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.036573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.037068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.037548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.040058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.046494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.048387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:05.064270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:05.065743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:05.066678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:05.069319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 267/559] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:05.070428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.073689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.084804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c19d-c80a-4d09-9938-97d36535a1df None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.123568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.124386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.124674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.124949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.136194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.137714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:05.157017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:05.158065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:05.158973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:05.158973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:05.158973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:05.158973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:05.162767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 293/560] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:05.163634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.186558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22225d93-a2bd-4850-acff-073c0c26443f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.225928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.227632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.228131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.228958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.244175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:05.253265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:05.254249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:05.255081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:05.257902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 268/561] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:05.258930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.288338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1c70-3132-48b8-b318-5d66cadca449 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.306203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.330175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.330850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.331800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.332106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.332996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:05.333309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.334746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.341284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.352728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 294/562] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:05.353858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.382121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa200-f55c-4ae0-9c15-eb26cec59e2a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.415853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.416535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.416816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.417089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.417803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:05.418108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.419504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360af88-9d9b-4995-b68d-c5df81189986 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.435324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 269/563] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:05.435324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a9116-34cb-4751-889a-17f5ecafebd6 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.435324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a9116-34cb-4751-889a-17f5ecafebd6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.435324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a9116-34cb-4751-889a-17f5ecafebd6 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.435324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 270/564] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:05.435324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a9116-34cb-4751-889a-17f5ecafebd6 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.460998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9441b-c093-4ef4-8e93-c8b68f9680ef None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.504272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.504957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.505277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.505558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.506295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:05.506616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.507960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb5dfd-9052-4025-aa09-7cb76833ac08 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.516118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 295/565] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:05.517452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aded3e-ed65-4daf-a3b0-2d6751bdae74 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.517745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aded3e-ed65-4daf-a3b0-2d6751bdae74 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.518022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aded3e-ed65-4daf-a3b0-2d6751bdae74 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.519392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 296/566] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:05.521298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aded3e-ed65-4daf-a3b0-2d6751bdae74 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.537632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a9116-34cb-4751-889a-17f5ecafebd6 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.557496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.560078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.572260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.573073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.573355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.573630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.583731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.585268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c2af6237-fd87-432f-a626-03a8f970f6ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:05.602725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:05.607968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 271/567] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de89d2-43f0-4214-97fb-5dd4f998966f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de89d2-43f0-4214-97fb-5dd4f998966f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.609072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de89d2-43f0-4214-97fb-5dd4f998966f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.624990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aded3e-ed65-4daf-a3b0-2d6751bdae74 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.626607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-32de89d2-43f0-4214-97fb-5dd4f998966f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:05.628261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-32de89d2-43f0-4214-97fb-5dd4f998966f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:05.659325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.660129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.660448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.660725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.667329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.668825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:05.683485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:05.684615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 297/568] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:05.685722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.701800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 272/569] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:05.703148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80d833-613d-478d-904b-80fe8326753d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.703456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80d833-613d-478d-904b-80fe8326753d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.703758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80d833-613d-478d-904b-80fe8326753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.727960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-bf80d833-613d-478d-904b-80fe8326753d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:05.729188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bf80d833-613d-478d-904b-80fe8326753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:05.782025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.788633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736365e2-76f2-4201-94fa-90bb9b70f021 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.796054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 273/570] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:05.796941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80d833-613d-478d-904b-80fe8326753d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.824386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.825182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.825463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.825736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.831808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:05.837499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.838971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:05.857255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:05.858257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:05.859133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:05.859133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:05.859133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:05.859133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:05.859133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 298/571] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:05.859422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.903971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80d833-613d-478d-904b-80fe8326753d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.940125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.940818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.941416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.941698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.942579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:05.942887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.944266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:05.955326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 274/572] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:05.956138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:05.961678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a025e-18fc-48ad-a7bf-d416c494a4ce None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:05.996104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:05.996790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:05.997076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:05.997356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:05.998133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:05.998449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:05.999869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.011231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 299/573] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:06.012034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.020689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.058566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.065023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fb0f0-ed16-4218-96c0-f48ef3b0f75c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.097627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.098430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.099028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.099613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.111160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:06.111643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8509877-be5a-4dde-82fe-743499ff3fbe None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:06.118982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:06.120110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:06.121431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 275/574] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:06.122226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.150042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.150834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.151119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.151392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.162218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:06.169996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:06.171087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 300/575] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:06.177271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.227424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c1b5-da2a-4726-b128-69462476b4b9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.237785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.261736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.262467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.262751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.263027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.263743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:06.264050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.268078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.274016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faed2a1-8964-4f7e-9ff3-ac422e4d5fe5 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.278775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 276/576] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:06.279621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.296219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.313531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.314257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.314541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.314821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.321608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.323126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:06.338441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:06.339476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 301/577] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:06.345976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.381557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34c4ca-cb2d-476f-a133-7c6432de275f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.415427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.416079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.416397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.416678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.417502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=85a16b32249c48eb89cf0d7cd55e5de7) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:06.417811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.419242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.424764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 277/578] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/role_assignments?user.id=85a16b32249c48eb89cf0d7cd55e5de7&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:06.425547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.440786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990b852-d779-4498-a016-71c9ff87a77d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.473356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.483305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.484042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.484358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.484647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.485465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:06.485777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.487206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.496976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 302/579] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:06.498265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.523894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.547132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e6aa-f690-4472-b53e-eff2ca416441 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.593589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.594197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.594564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.595345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.596117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:06.596474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.597884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.606750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 278/580] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:06.607634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.629287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde55e9-dd04-4652-aff5-27fca89d8865 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.679227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.680015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.680307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.680600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.688857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.690305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:06.705535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:06.706546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 303/581] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:06.707584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.730456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fc9c1-929c-4ca8-b5af-eb544193a0c8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.758430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.769337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.770271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.770728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.771170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.772118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:06.774458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.774458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.783588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:06.792318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 279/582] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:06.792318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.816451 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b91d65-d11a-4c5e-b098-60d9a7d6da99 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.853139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.853853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.854130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.854402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.865590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.867035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:06.886471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:06.887418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:06.888343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:06.888343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:06.888343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:06.888343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:06.888343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 304/583] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:06.889399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:06.905794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fee72d-1252-4810-ab53-6072bee19087 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:06.947616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:06.948447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:06.948734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:06.949009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:06.959981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:06.961478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:06.980562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:06.981792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:06.982708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:06.982708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:06.982708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:06.982708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:06.982708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 280/584] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:06.983004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.003874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.006794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4370d-a3eb-4d48-b53d-c82a05fdd9dc None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.042724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.044112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.045157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.045751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.046501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.048272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:07.048608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.050198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.068659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 305/585] 10.222.0.46 () {56 vars in 1137 bytes} [Sat Jun 4 09:05:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:07.073409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.096022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd722a-37c1-4d76-94a1-fda076bc56b7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.138361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.139455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.139990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.140278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/5c1d511f4c6341e09d54bf8709efdf8e/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.161359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=5c1d511f4c6341e09d54bf8709efdf8e, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.162868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.182521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54324a4c-c484-42dd-82dc-939a63b3f8bc None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.216448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.217097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.217376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.217643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.218349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:07.218649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.220043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.231404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 306/586] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:07.232208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.256314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 281/587] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:06 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/5c1d511f4c6341e09d54bf8709efdf8e/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:07.256314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.263404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.368739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68761dfb-4e4f-48f9-a348-3abf3c6b35dd None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.399854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec63838-f5ec-414c-8b41-c477398dfcb4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.402125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.410995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.412150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.418318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.418318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.418318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:07.418318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.418318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.429968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 307/588] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:07.431234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.444354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.445671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.446363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.446874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.463463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.465240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:07.489332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:07.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:07.491183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:07.491183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:07.491183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:07.491183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:07.491183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 282/589] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:07.494423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.544464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.573335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d896-30fe-40a2-8693-1490d9034a09 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.613443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.614726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.615215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.615710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.619753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44c01-d995-49df-9626-78ce8b56481b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.624796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.626563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:07.643716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:07.644833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 04 09:05:07.645748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:07.648694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 308/590] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:07.649754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.661663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.662059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.662343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.662619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.670726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.671900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:07.688191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:07.689232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:07.690159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 283/591] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:07.690752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.710741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.767857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee763c2-6b2c-4d94-be60-fb669c22c10e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.801230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3f762-f0aa-4ea7-83ae-9cdc6c42e60d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.804521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.805457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.806070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.806562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.807494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:07.808005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.809608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.816671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.822273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 309/592] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:07.823205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.855703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.856778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.857238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.857659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.858641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:07.859126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.861071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.885491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 284/593] 10.222.0.46 () {56 vars in 1137 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:07.886671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:07.927772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595cd71-e357-441d-8eb5-390fcb3ffb26 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:07.962492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:07.963115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:07.963402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:07.963682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:07.964554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:07.964866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:07.966311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:07.966740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:07.986003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 310/594] 10.222.0.46 () {56 vars in 1139 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:07.986858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.003223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ece83-b2f6-4b0c-83bf-0275f0694a9a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.043533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.044863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.045365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.045869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.047054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.054896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.056708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:08.071609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:08.072703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:08.073799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:08.076371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 285/595] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:08.076955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.092015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76c5fa-95da-4406-a065-62ac7874ff69 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.130749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.131553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.131838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.132467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.143902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.145444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:08.164248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:08.165226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:08.166106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:08.166106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:08.166106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:08.166106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:08.166106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 311/596] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:08.166106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.201433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e096f-6ec0-4396-a561-93c7edf6ac9d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.235850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.243082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.244175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.244743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.245223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.258467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.260403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.271816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297e962-ece8-4fac-a560-64bc61ee882f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.281718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:08.283447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:08.284576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:08.285581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:08.285581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:08.285581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:08.285581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:08.287711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 286/597] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:08.289398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.307825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.308580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.308877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.309153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.310028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:08.310247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.311776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.325540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 312/598] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:08.326424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.413532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e5f18-e575-4953-9726-384e37bc5481 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.435826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090a860-e365-4360-9bf9-75a134560ff1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.452117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.453188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.454347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.454834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.455860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:08.456456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.458096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.473026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 287/599] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:08.474229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.474542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.474957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.475246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.475550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.476248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:08.476621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.478065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.491594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 313/600] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:08.492507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.522221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.529353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.586290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c58c7a-f831-4e88-9ff7-65e07692b119 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.597327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1471e-9ea9-43e0-af61-6d3e8b32800d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.624735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.626079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.626602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.627097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/e53e659f7faf403b9f33f393d166dda3/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.634413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.635224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.635510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.635794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.644170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.645951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.646825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=e53e659f7faf403b9f33f393d166dda3, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.648525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:08.661641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:08.662655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:08.663569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 314/601] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:08.664543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.756980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.769048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab8f46-c5cc-442c-b270-a797051e0316 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.794849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 288/602] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:08 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/e53e659f7faf403b9f33f393d166dda3/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:08.795835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.815989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.817251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.818277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.823635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.827179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.829711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-e9bd3aff-9165-40c8-8c31-16995dc4b0b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:08.850713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:08.851691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:08.852678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:08.855341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 315/603] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:08.857203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4967714-6d7c-423c-8bc8-2480e9fb81eb None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.857785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4967714-6d7c-423c-8bc8-2480e9fb81eb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.858266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4967714-6d7c-423c-8bc8-2480e9fb81eb None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.860072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 316/604] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:08.861152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4967714-6d7c-423c-8bc8-2480e9fb81eb None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:08.909985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:08.917519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b18c8e-4365-4d48-85e2-32cd0ff6c0b7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:08.955759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:08.956671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.957092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.957476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.958397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:08.958812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:08.960437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14036b9-bdff-4513-aada-bbf3398f5b0b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:08.974092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 289/605] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:08.975628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:08.976021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:08.976429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:08.994097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:08.995499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:08.998182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4967714-6d7c-423c-8bc8-2480e9fb81eb None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:09.046888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:09.047517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.047805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.048134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.049029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:09.049353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:09.050795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:09.074209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 290/606] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:09.075046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:09.080439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 317/607] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:09.084497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:09.097941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:09.185483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:09.207363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9c0bb-ce7d-4eee-b1b6-c590e52dcb2a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:09.226254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e167a-efea-4807-81c9-81756d5fe7a3 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:09.245911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:09.246965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.247209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.247537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/8b3a9e1e2e3d438f92286a4158c8d4cd/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.262963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=8b3a9e1e2e3d438f92286a4158c8d4cd, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:09.264440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:09.278124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:09.279012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.279287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.279588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/e7837f3811b04542a6e7f56e7fa118d3/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.299968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=e7837f3811b04542a6e7f56e7fa118d3, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:09.301370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:09.384317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 291/608] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:09 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/8b3a9e1e2e3d438f92286a4158c8d4cd/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:09.384317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:09.417398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 318/609] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:09 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/e7837f3811b04542a6e7f56e7fa118d3/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:05:09.506680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da883137-0656-45ab-b46d-b008a0eb3a15 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:09.558839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26850200-1039-43df-a386-2da5678ba3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:09.559586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26850200-1039-43df-a386-2da5678ba3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.559868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26850200-1039-43df-a386-2da5678ba3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.560144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26850200-1039-43df-a386-2da5678ba3b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.570329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:09.578280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26850200-1039-43df-a386-2da5678ba3b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-26850200-1039-43df-a386-2da5678ba3b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:09.586877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:09.588252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:09.589135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:09.592895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 292/610] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:09.599388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:09.641196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0724658-e92a-422f-bec9-cf5c6a85b55f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.641618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0724658-e92a-422f-bec9-cf5c6a85b55f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.641807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0724658-e92a-422f-bec9-cf5c6a85b55f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.644086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 293/611] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:09.658239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.658583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.658807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.680510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:09.682306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:09.693232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:09.758617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43e42d-c4fc-485e-8aa9-cd559abf09b4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:09.787560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 294/612] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:05:09.808342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:09.809894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:09.813690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.813690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.813690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.816523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:09.817110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:09.819028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:09.830708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 319/613] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:09.838246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:09.928314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36b993-1195-4636-b0f2-49fdfe28ecb2 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:09.950995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:09.970744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:09.976856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:09.976856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:09.976856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:09.993338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:09.998749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9cbb0-7987-47fc-bb8e-6ab8776844ce None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:10.012269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:10.013312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:10.014141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:10.021662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 295/614] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:10.024842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.034243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:10.066503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.067876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.068582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.069149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.086830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.088693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:10.116777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:10.117730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:10.118598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:10.118598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:10.118598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:10.118598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:10.124783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 320/615] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:10.129963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.146945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1dd83-07c1-42f4-94ae-e5fcb5e1664e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.183533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.184387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.184770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.185151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.186010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:10.186423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.188115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.197547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 296/616] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:10.203075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.254988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:10.305357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bea52-bf2f-45f6-b6f9-866630af8686 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.313280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f483d-bbe3-4b7c-8355-5f128bd69d97 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.349469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.350359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.350747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.351130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.363446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.365206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:10.383825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:10.385124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:10.386208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:10.386208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:10.386208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:10.386208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:10.386506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.386506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.386506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.386506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.387913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:10.387913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.388085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 297/617] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:10.388734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.401425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.422900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 321/618] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:10.428373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.491645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:10.563842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:10.569379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc9a7e-08af-4a4a-b895-c27b7e067de8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.618267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454685d-fded-42d5-b6f9-cbcb0f18df5b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.634685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.635413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.635698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.635974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.636735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:10.637039 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.638419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.663103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 298/619] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:10.668869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.690114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.690114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.690114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.690480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.702578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.704571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:10.735340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:10.736367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:10.737315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:10.743011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 322/620] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:10.746978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.857515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a9b3-b9d0-4fd7-832f-4207bc457797 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.880990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:10.897712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44388a2a-50f3-49fa-8f2b-e0cbc0600108 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:10.934851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.935845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.936135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.936494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.945923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.947377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:10.964909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:10.965921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:10.966868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 299/621] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:10.971850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:10.974140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:10.975349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:10.975777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:10.977187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:10.978264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:10.978772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:10.980459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.001198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 323/622] 10.222.0.46 () {56 vars in 1135 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:11.007463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.018274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:11.125108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5cf55-076b-48a7-812f-cdbe4626ad50 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.169012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:11.171159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.173024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.174676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.186477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e9de9-b8c3-49b7-820c-fde2b4e79854 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.198611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:11.202076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:11.233843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:11.234831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:11.236360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:11.236360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:11.236360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:11.236360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:11.250119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 324/623] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:11.254790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.258376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:11.261442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:11.262126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.262408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.262681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.263520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:11.263827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:11.265285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.292237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 300/624] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:05:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:11.297413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.378128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:11.402764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a60c-9bdb-4dd7-b5b3-14eeddb62add None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.454437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070dd44-2cf0-4342-aa0f-8913420aa372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:11.455482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070dd44-2cf0-4342-aa0f-8913420aa372 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.456469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070dd44-2cf0-4342-aa0f-8913420aa372 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.456948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070dd44-2cf0-4342-aa0f-8913420aa372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.482833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7b2b-c452-40b3-a59f-339b4b632e55 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.532884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:11.534723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.535226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.535938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.537588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:11.538101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:11.539970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f18a1c-8ce8-483a-ac9e-c6d4dfa12bc7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.543051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070dd44-2cf0-4342-aa0f-8913420aa372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:11.544527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070dd44-2cf0-4342-aa0f-8913420aa372 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.558928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 325/625] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:11.561587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdeafd-f1eb-4763-9cb0-21bb7783990d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.562112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdeafd-f1eb-4763-9cb0-21bb7783990d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.562586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdeafd-f1eb-4763-9cb0-21bb7783990d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.564464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 326/626] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:11.571147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdeafd-f1eb-4763-9cb0-21bb7783990d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.642911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:11.662050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:11.667530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 301/627] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:11.668942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.669229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.669914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.690995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:11.692173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:11.718256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdeafd-f1eb-4763-9cb0-21bb7783990d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.769626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:11.770730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.771106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.771478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.772772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 302/628] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:11.773954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.794546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:11.796272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:11.816535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:11.817533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:11.818478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:11.820996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 327/629] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:11.826320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.902190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672738df-580d-4f5d-bebc-7c835956e99a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.910397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:11.940708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:11.942424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:11.942941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:11.943412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:11.957651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:11.959043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771c5c-2dee-4b37-bbb7-43dd0f46369a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:11.960103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:11.987027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:11.987946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:11.988909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:11.988909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:11.988909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:11.988909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:11.995328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 303/630] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:11.995328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:11.998344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.012091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.013277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.013719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.019487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.026160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:12.034289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:12.035225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:12.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:12.038866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 328/631] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:12.039869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.115010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67bf3a-24ce-4163-aca0-eed664cba005 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.148019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee5857-164c-4a5f-aac0-e695dd72dda1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.152513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.153181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.153465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.153741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.155052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:12.155370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.156855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.169719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 304/632] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:12.170565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.189880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.190792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.191074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.191347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.192060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:12.192390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.193192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.193751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b72843-cfa8-4b7c-85ca-6c9dbcdd0788 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.206508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 329/633] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:12.212065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096eda1-294a-43b9-8a55-24967b0a8e83 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.212380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096eda1-294a-43b9-8a55-24967b0a8e83 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.212662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096eda1-294a-43b9-8a55-24967b0a8e83 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.214070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 330/634] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:12.214657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096eda1-294a-43b9-8a55-24967b0a8e83 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.261596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.298460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b0006-ce29-451d-839e-7c2b39ef3f98 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.324502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096eda1-294a-43b9-8a55-24967b0a8e83 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.334661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.335338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.335622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.335891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.336702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:12.337011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.338391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.347575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 305/635] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:12.348251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.359878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aaa445-9ac7-4adf-92bb-4c40594ce400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.360823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aaa445-9ac7-4adf-92bb-4c40594ce400 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.361100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aaa445-9ac7-4adf-92bb-4c40594ce400 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.361372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aaa445-9ac7-4adf-92bb-4c40594ce400 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/85a16b32249c48eb89cf0d7cd55e5de7/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.377610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aaa445-9ac7-4adf-92bb-4c40594ce400 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=85a16b32249c48eb89cf0d7cd55e5de7, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.379054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aaa445-9ac7-4adf-92bb-4c40594ce400 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.427164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.445932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3109b-4fca-42eb-9976-e3ca45eda87d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.475674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 331/636] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:12 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/85a16b32249c48eb89cf0d7cd55e5de7/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:12.477126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.477413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.477688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.479382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.480202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.480515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.480791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.488698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.490232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.495643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:12.497187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6c03a32f-307d-4474-a6e1-a4907b6cf40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:12.506353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:12.509969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:12.510859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:12.513842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 306/637] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:12.515735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc2e1-3e0b-49ca-b304-4234d1a145a5 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.516217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc2e1-3e0b-49ca-b304-4234d1a145a5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.517274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc2e1-3e0b-49ca-b304-4234d1a145a5 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.519980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 307/638] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:12.519980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc2e1-3e0b-49ca-b304-4234d1a145a5 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.590191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 332/639] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:12.591124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.636127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.653970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc2e1-3e0b-49ca-b304-4234d1a145a5 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.702140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.702140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.702140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.702140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.714436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.717079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.719070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-626e5658-d792-4f0a-b36d-3ca2298412c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:12.741034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:12.742006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:12.743621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:12.743621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:12.743621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:12.743621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:12.743621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 308/640] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:12.744426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.744717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.744999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.755943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12615033-a9b6-47ce-b271-a9b9f0588597 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.764511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:12.765641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:12.796794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.798166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.798465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.798746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.799687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:12.800024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:12.801504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:12.815004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 333/641] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:12.815660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.847730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 309/642] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:12.851104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:12.908345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:12.927824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f079a4-b9e8-45b3-b52a-b0c759e719f7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.963657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:12.964631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:12.964917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:12.965195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:12.968655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3826c83-313f-499d-9898-e83aafb57c90 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:12.976159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:12.983968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:12.984994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 334/643] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:12.986597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.000113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.018582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.018582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.018582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.018582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.018956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:13.018956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.018956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.035119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 310/644] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:13.035119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.105558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18984d4-dcd5-4a3d-9023-055a654b8b33 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.140495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.141249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.141628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.141978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.142885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=5c1d511f4c6341e09d54bf8709efdf8e) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:13.143271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.144428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dbc56-d127-4d90-b5c3-fe019879f6c5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.144888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.152030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 335/645] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:12 2022] GET /identity/v3/role_assignments?user.id=5c1d511f4c6341e09d54bf8709efdf8e&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:13.152851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.163793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.198220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.199314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.200352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.200864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.217993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:13.230273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:13.231496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:13.232710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:13.236600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 311/646] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:13.239428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.283488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b0892-0b5c-40c5-9793-3a1fda9a62e9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.299711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.331324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.332529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.333029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.333642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.334617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:13.335230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.337510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.351712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 336/647] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:13.352708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.375059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d337bd3-f400-437d-a068-72ada36c1ff6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.418774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.419772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.420059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.420362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.430778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.433998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.435939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:13.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:13.452033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 312/648] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:13.455116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.487052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844620c-eb61-494b-880c-8bd4d3796aab None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.536832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990e95-2582-40be-98d4-507bc393223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.538332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990e95-2582-40be-98d4-507bc393223b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.538626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990e95-2582-40be-98d4-507bc393223b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.538905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990e95-2582-40be-98d4-507bc393223b None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.539664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990e95-2582-40be-98d4-507bc393223b None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:13.539968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990e95-2582-40be-98d4-507bc393223b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.541392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990e95-2582-40be-98d4-507bc393223b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.552851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 337/649] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:13.553964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990e95-2582-40be-98d4-507bc393223b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.571742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be6938-8c43-46bd-b77b-64dfa9743fa7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.573473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.613146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.613950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.614237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.614549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.627625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.629530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:13.651210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:13.652403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:13.653508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:13.653508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:13.653508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:13.653508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:13.655056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 313/650] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:13.659021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.687094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990e95-2582-40be-98d4-507bc393223b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.726880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.734645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.739324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.739324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.739324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.739324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:13.739324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.740758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.767068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 338/651] 10.222.0.46 () {56 vars in 1137 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:13.770272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.775231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313770f-7bec-4ddb-a008-a726de979953 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.815838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.816628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.816959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.817241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.829584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.831056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:13.850692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:13.851648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:13.852579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:13.852579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:13.852579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:13.852579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:13.852579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 314/652] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:13.853270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:13.863274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:13.914716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a02c1-a355-4db6-aa03-ffd8a7ebd5d8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.961914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:13.962924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:13.963360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771f83a-98b8-4c0d-8da8-1a6227f0364d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:13.963656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:13.964133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:13.966670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:13.967198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:13.968820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:13.985813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 339/653] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:13.987010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.007721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.008172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.008542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.008812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.052750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.086358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.088094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.099174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.116253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171835-f30e-4172-a8e4-b0cf11e8f34b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.154191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.155304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.155792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.156269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.158272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:14.158904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.160615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.173773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 340/654] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:14.174911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.202127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 315/655] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:14.203110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.285199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9915239-df87-49e5-a129-4e2f81ce2ba7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.295492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.323604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.324609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.325010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.325275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5ff2e-15ef-4263-92e3-ca8ca719bc38 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.325469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.334755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.336420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.338340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:14.358059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:14.365568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 341/656] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:14.366505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.373738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.374974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.375465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.375990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.385269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.387158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:14.405171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:14.406179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 04 09:05:14.407689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:14.410599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 316/657] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:14.411835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.504382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3815b-1c76-4de4-9164-318dc66aa3c1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.528517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84faa03b-3919-4365-a0dd-b867208c0f7c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.539167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.540392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.540686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.540962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.555784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.557940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.567966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.568841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.569627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.570009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.571230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:14.571644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.573220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:14.580246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:14.581343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:14.582740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:14.582740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:14.582740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:14.582740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:14.582740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 342/658] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:14.583349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.596109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 317/659] 10.222.0.46 () {56 vars in 1137 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:14.596974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.613655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.630383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.711411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305eb07-6cc8-4478-9ed6-d1d4a638a250 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.727295 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae6705-9a1e-47a0-bc39-2e6d01e29439 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.755861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.756993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.757492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.758204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.759494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:14.760111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.762329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.770342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.771124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.771478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.771825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.772953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:14.773332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:14.774955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:14.790203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 343/660] 10.222.0.46 () {56 vars in 1139 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:14.790203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.795021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 318/661] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:14.796330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:14.877500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.887339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:14.899512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3983b-c3b4-4810-bd1a-e227ea7ec1f8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.903507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02fb59-7311-4a12-bd3b-12514172b626 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:14.935373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.936024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.936332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.936615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:14.939390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4180c-dc78-4e32-8ee8-28e66407cf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:14.940229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4180c-dc78-4e32-8ee8-28e66407cf0e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:14.940873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4180c-dc78-4e32-8ee8-28e66407cf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:14.941146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4180c-dc78-4e32-8ee8-28e66407cf0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.008275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.009868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.015522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4180c-dc78-4e32-8ee8-28e66407cf0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.017057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4180c-dc78-4e32-8ee8-28e66407cf0e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.095184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.104459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.121163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 344/662] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:15.121163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.126933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 319/663] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:15.131515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531654-8827-4571-bab1-47215c574fbe None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.131515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531654-8827-4571-bab1-47215c574fbe None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.131515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531654-8827-4571-bab1-47215c574fbe None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.131515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 320/664] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:15.131515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57531654-8827-4571-bab1-47215c574fbe None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.240263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57531654-8827-4571-bab1-47215c574fbe None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.244716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db07a0-e138-40cb-904d-354c960ba475 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.286143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.287003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.287289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.287509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.292584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.293542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.293820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.294092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.298351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.299833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.306981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.308473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:15.314127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:15.315026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:15.315918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 345/665] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:15.317403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-b27fb615-2441-4320-88d5-706cb7585ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:15.331776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:15.332748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:15.333672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:15.333672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:15.333672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:15.333672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:15.333672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 321/666] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:15.338117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.338409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.338968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.356634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:15.357811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:15.361364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.390060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.428276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 322/667] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:15.428517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24538-49ec-44d5-a2f8-2845bbf65473 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.429605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.477142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.478122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.478502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.478870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.493484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.495148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:15.513682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:15.514766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:15.515792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:15.515792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:15.515792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:15.515792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:15.517427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 346/668] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:15.518467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.535059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9cb1-795a-41e8-837b-8e6a936f4074 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.570721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.571385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.571664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.571937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.572747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:15.573053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.574497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.592095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 323/669] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:15.592095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.619901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.620530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.634365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9df21c-fb9e-4060-9894-0e137ba11bb6 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.668742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.669491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.669774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.670049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.670767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:15.671072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.672468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.684800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 347/670] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:15.685612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.698868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f298d0-1220-400f-8300-943eef0e555c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.733643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.734436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.734717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.734993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.745387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:15.753505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:15.754453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:15.755284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 324/671] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:15.755994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.789695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfdacc-94a1-4027-a9fd-7dbb9500f303 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.823632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.824317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.824601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.824885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.825582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:15.825889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.827312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77eab-d096-4864-85e4-f81bbff2493e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.839359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 348/672] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:15.840684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4d174-7237-439c-a1e1-ae51fcb80385 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.840968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4d174-7237-439c-a1e1-ae51fcb80385 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.841240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4d174-7237-439c-a1e1-ae51fcb80385 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.842640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 349/673] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:15.843156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4d174-7237-439c-a1e1-ae51fcb80385 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.851184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.859520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:15.866859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c39ef-7c85-44d1-8dce-024a493a528f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.902172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.902825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.903124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.903416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:15.904446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=e53e659f7faf403b9f33f393d166dda3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:15.904756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:15.906200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:15.913093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 325/674] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/role_assignments?user.id=e53e659f7faf403b9f33f393d166dda3&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:15.913972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:15.958166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4d174-7237-439c-a1e1-ae51fcb80385 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:15.997005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:15.997735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:15.997992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:15.998274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.007185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.008746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:16.024654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:16.025804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 350/675] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:16.026735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.046472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0df8-7958-4e3e-832c-d64dba32c7b1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.088713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.089545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.089908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.090284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.091355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.091471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.092857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3742f03-7498-47bf-b84b-503455f5e44e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.107867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 326/676] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.107867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.107867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.107867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.111428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.125454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:16.126595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:16.134682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.153711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61319fa-ef05-4852-a00c-a4c75ed7090f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.193312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.194107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.194397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.194662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.202314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.208437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 327/677] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:16.208580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.208671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:16.221480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:16.222580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:16.223706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:16.224516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 351/678] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:16.225773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.327996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591df68-5838-4684-a6dc-d15e6f259a02 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.339995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223e09e-6e03-4807-9a3a-5dc4b0d104de None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.364426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.365364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.365921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.366291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.367174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.367577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.369287 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.370719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.379921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.380767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.381052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.381331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.385403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 328/679] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.390772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.393206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.394505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.394662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:16.413710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:16.414837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:16.416034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:16.416034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:16.416034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:16.416034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:16.420007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 352/680] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:16.420339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.502079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ddb3-daaa-4b30-b622-d27ee5c0b1d0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.530509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f2474-a063-4b72-9c02-da602b034e92 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.541111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.542188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.542474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.543030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.543832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.544146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.546368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.569328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 329/681] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.569328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.571842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.572755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.573110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.573480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.585950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:16.593692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:16.594772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:16.595722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 353/682] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:16.596599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.647116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.655473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.691206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf555-e28a-49e8-941a-981d13e4ac75 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.716913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5d1e5-811c-47e5-a0de-0b0fa4f5222a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.730648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.731325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.732079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.733334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.733334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=8b3a9e1e2e3d438f92286a4158c8d4cd) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.733615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.736043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.752577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 330/683] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/role_assignments?user.id=8b3a9e1e2e3d438f92286a4158c8d4cd&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.752577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.757792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.758467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.758749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.762842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.762842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.762842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.762842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.777160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 354/684] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.778171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.884633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7beca-ed08-4ca9-a306-8b3f672f020f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.915587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.925628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.926355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.929522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.929522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.929522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=e7837f3811b04542a6e7f56e7fa118d3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.929522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.929813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d830d-67ca-45c2-8359-b1b18a8e530f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:16.930047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:16.930153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.941243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 331/685] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/role_assignments?user.id=e7837f3811b04542a6e7f56e7fa118d3&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.942236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:16.967358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:16.968154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:16.968488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:16.968778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:16.970074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:16.970397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:16.971829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:16.981843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 355/686] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:16.988471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.072775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbecae3-a522-4b7a-9bf4-321df000dc3b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.105634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7438b-871a-4ec7-938b-8ab562b6d67c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.115425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.116249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.116607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.116918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.126524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.128000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:17.145268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:17.147291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:17.148868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:17.150151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.150319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 332/687] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:17.150491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.150491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.150491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.154900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.161517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.162998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:17.181656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:17.183784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:17.185959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:17.185959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:17.185959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:17.185959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:17.185959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 356/688] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:17.188711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.208783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:17.218885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:17.284019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf723a47-07a9-434b-8455-67202ae56dd4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.312131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d03d89-3efd-4756-b159-4fdb7f2bff48 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.326043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.327073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.327574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.328148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.330236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:17.330812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.332712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.354466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.355046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 333/689] 10.222.0.46 () {56 vars in 1135 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:17.359250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.359582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.359869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.360706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:17.361021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.362032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.362453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.380931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 357/690] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:17.386220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.523468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:17.541010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1f6c-5fc4-453b-9531-d779afd18bc9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.558908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:17.571740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8377b-b698-48a5-bdf7-98d246a9a219 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.612788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.613776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.614289 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.614769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.636908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.638096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.638610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.639093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.652448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.654264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:17.679492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:17.680532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:17.681575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:17.685883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 358/691] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:17.691005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.765413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.767198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.835863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444b4ea-b3e9-4ce4-9f80-7cfc0276b307 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:17.876276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:17.876948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:17.877313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:17.877589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:17.878429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:17.878787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:17.882777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:17.882927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:17.900492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 359/692] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:17.908093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:17.993805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 334/693] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 04 09:05:17.999517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.019950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.024610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b552-fd72-4b60-aba2-d9827b57488c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.071720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.072451 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.072734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.073008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.149673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.158898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.160780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.213187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6286032-52e7-47cb-a37d-8edeb9dda9cf None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.286672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 360/694] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 04 09:05:18.291562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.311530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.312357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.312640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.312911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.336960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.338445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:18.380897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:18.381867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:18.382764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:18.382764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:18.382764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:18.382764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:18.382764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 335/695] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:18.393314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.412419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f33f4-5871-47d7-9b4f-2491def198a7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.432535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.467864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.469089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.469610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.470091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.481933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.483910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.484426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b509b955-0648-47fb-b1a9-314076c6b44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:18.506166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:18.507113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:18.508007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:18.508007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:18.508007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:18.508007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:18.511306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 361/696] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:18.513097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8970102-6a42-4e10-9884-793b4dbd2f1f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.513630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8970102-6a42-4e10-9884-793b4dbd2f1f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.514110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8970102-6a42-4e10-9884-793b4dbd2f1f None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.520542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 362/697] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:18.521605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8970102-6a42-4e10-9884-793b4dbd2f1f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.529203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070d28c-b047-45c1-94d4-f0d485d87007 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.565315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.565992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.566274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.566551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.567281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:18.567585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.569013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bd25a-7640-4696-bea1-b0031a20be61 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.586100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 336/698] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:18.586100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.586100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.586100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.605117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:18.606395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:18.655981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8970102-6a42-4e10-9884-793b4dbd2f1f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.688731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 337/699] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:18.688731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.707662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.708837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.709330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.709908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.710910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:18.711488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.713345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.736516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.738791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 363/700] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:18.740789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.753044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.789748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f102a43-15ea-4d70-a782-b7882bcc5bf2 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.825958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.827466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.827747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.828017 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.835925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.837517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:18.852316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:18.853269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:18.854157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 338/701] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:18.854673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.856878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd28cb5-0f50-47c6-a8cb-4b0fb68fa867 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.894076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.895255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.895764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.896279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:18.897244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:18.898834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:18.913974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:18.915057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:18.915920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:18.924008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 364/702] 10.222.0.46 () {56 vars in 1129 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:18.925171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:18.960267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb290fe-3d69-4a9a-825a-d70a9476844d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:18.972769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:18.998205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:18.999007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:18.999307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:18.999622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.001638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:19.007793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.009332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:19.026028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:19.026994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:19.027918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 339/703] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:19.028469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.041091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9d6fc-760e-4f77-8796-b52b87a19f1d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.078809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.079899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.080482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.081077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.082233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:19.082747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.084413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.098579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 365/704] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:19.099813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.142264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790b285-c690-4da2-a69e-3d5089c1bf45 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.183073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.183744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.184024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.184309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.185099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:19.185431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.186955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.197228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 340/705] 10.222.0.46 () {56 vars in 1124 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:19.197228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.221869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6361-0150-468a-9cff-61df846ecc64 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.242333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:19.260401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:19.272941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.273991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.274482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.275008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.276026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:19.276569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.278284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.298362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 366/706] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:19.299738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.320266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d190f-b46a-4d91-b61a-b8d726101ee1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.358725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.362358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.362358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.362358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.362358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=85a16b32249c48eb89cf0d7cd55e5de7) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:19.362358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.363895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.371529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 341/707] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/role_assignments?user.id=85a16b32249c48eb89cf0d7cd55e5de7&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:19.372692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.425372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d20130-6235-4483-88f1-444686ee3ddf None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.466769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.467824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.468355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.468854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.469914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:19.470427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.472117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ecf50-cfcc-4c22-8193-5c01ea284cb6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.479427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 367/708] 10.222.0.46 () {56 vars in 1124 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:19.513523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:19.518965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe09b7-db9d-4e5e-a280-3ff6898af903 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.560578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.561371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.561654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.561926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.562808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=5c1d511f4c6341e09d54bf8709efdf8e) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:19.563119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.564660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbf15f-5249-40c2-8fc0-5cdfad4274b4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.571708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 342/709] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/role_assignments?user.id=5c1d511f4c6341e09d54bf8709efdf8e&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:19.588812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564164e-be25-4a42-8199-69d216a80ccd None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.589589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564164e-be25-4a42-8199-69d216a80ccd None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.590109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564164e-be25-4a42-8199-69d216a80ccd None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.592268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 368/710] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:19.597843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7dea0-c34a-409d-9d12-95deb40f695e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.598141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7dea0-c34a-409d-9d12-95deb40f695e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.598408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7dea0-c34a-409d-9d12-95deb40f695e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.623417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-27b7dea0-c34a-409d-9d12-95deb40f695e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:19.624729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-27b7dea0-c34a-409d-9d12-95deb40f695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:19.728474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 343/711] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:19.752308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564164e-be25-4a42-8199-69d216a80ccd None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.755486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad9e83-4129-4fd6-812c-e2cbb278b276 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.755785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad9e83-4129-4fd6-812c-e2cbb278b276 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.756071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad9e83-4129-4fd6-812c-e2cbb278b276 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.758146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 344/712] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:19.764159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.764800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.765282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.789909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:19.791163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:19.843439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:19.899722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 345/713] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:05:19.914844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564164e-be25-4a42-8199-69d216a80ccd None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:19.926741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:19.951394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:19.953562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:19.955477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:19.956087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:19.956542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] PATH_INFO: `/v3/services/network` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:19.957886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:19.959712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:19.979527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:19.980856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:19.981992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:19.985003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 369/714] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:19.991880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:20.045601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27cc6-efec-4966-b45f-fe409aabb088 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:20.084976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:20.086377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:20.086871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:20.087382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:20.099530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:20.101499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:20.122952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:20.123941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:20.124896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:20.124896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:20.124896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:20.124896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:20.132434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 346/715] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:20.134003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:20.187250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81c5fd-87bd-4129-8ebd-06b4fbaf256e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:20.216417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:20.224570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:20.244953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e827abd-8c20-48c0-9c65-4550d3c2333b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:20.269584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:20.270268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:20.270593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:20.270848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:20.271545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] RBAC: Adding query filter params (name=network) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:20.271852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:20.273265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:20.284952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 370/716] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:20.289324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:20.312845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:20.315104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:20.316188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:20.316736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:20.317706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:20.320009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:20.321695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:20.355963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 347/717] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:20.373456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:20.441696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bffa42-a0e7-4717-85e3-68c0ed3bc19c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:20.485095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:20.486710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:20.487198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:20.487686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:20.488636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] RBAC: Adding query filter params (type=network) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:20.489280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:20.490929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af731-9794-4f72-a0af-beb794f7860e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:20.496714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 371/718] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:20.594416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709252-274a-4e92-a6c6-63c031812ca1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:20.647055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e7251-eb70-427b-99ca-39a103310e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:20.647758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e7251-eb70-427b-99ca-39a103310e6f None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:20.648038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e7251-eb70-427b-99ca-39a103310e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:20.648327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e7251-eb70-427b-99ca-39a103310e6f None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:20.649056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e7251-eb70-427b-99ca-39a103310e6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:20.650521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e7251-eb70-427b-99ca-39a103310e6f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:20.663020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:20.765297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 348/719] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:05:20 2022] POST /identity/v3/users => generated 254 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:22.250082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583710-dc7c-4b6c-80dd-8a613f76bddc None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.250612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583710-dc7c-4b6c-80dd-8a613f76bddc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.250612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583710-dc7c-4b6c-80dd-8a613f76bddc None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.252677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 372/720] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:22.258002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.258512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.258601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.280884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:22.282101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:22.338743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64931dd8-d3d1-4317-a094-3ac926390a69 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.339359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64931dd8-d3d1-4317-a094-3ac926390a69 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.339855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64931dd8-d3d1-4317-a094-3ac926390a69 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.341670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 373/721] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:22.347654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e0777-f158-4375-b8a8-f30f7ddfb224 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.348215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e0777-f158-4375-b8a8-f30f7ddfb224 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.348717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e0777-f158-4375-b8a8-f30f7ddfb224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.368032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e79e0777-f158-4375-b8a8-f30f7ddfb224 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:22.368467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 349/722] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:05:22.369933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e79e0777-f158-4375-b8a8-f30f7ddfb224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:22.403537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:22.438344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:22.465557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 374/723] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:05:22.474230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9349f7a-e374-4ecb-943d-f5d3d651699d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.474793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9349f7a-e374-4ecb-943d-f5d3d651699d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.475394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9349f7a-e374-4ecb-943d-f5d3d651699d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.477220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 375/724] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:22.490792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.491455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.491964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.516003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3e80f-a977-48a9-a8a0-5242fa24ba6f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:22.518958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:22.520393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:22.538218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:22.554756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:22.555703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.555993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.556275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.569767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:22.578685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:22.579679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:22.580539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 350/725] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:22.582286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:22.603844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 376/726] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 04 09:05:22.605122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:22.693675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81987c22-96e6-490b-b0cb-15fe0eaa9e04 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:22.726446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fff3d-9c48-4893-9a45-370f2e6243fb None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:22.733644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:22.734481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.734767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.735051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.738451 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:22.750676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:22.759932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:22.761247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:22.762098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:22.764933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 351/727] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:22.766147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:22.783516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:22.785050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.786293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.786868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.793016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:22.793016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:22.793016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:22.804342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 377/728] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:22.805417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:22.824551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:22.904516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ca08-e15b-4718-ad3f-50736a857b11 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:22.935333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f1248-379b-4414-83c9-1ff8d697ef13 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:22.946620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:22.947573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.948080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.948580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.949510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:22.951079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:22.974112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:22.974817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:22.975261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:22.975388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:22.976356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:22.976673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:22.978059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:22.992613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 378/729] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:22.992613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.027143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:23.034200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 352/730] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:22 2022] POST /identity/v3/services => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:23.035481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.136221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa029ea9-9ad7-4175-96ec-80e387faf15c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.161415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:23.179078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e458b-ab62-4459-88f2-168a9be7d9cd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.183978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.184951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.185335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.185680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.198048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.199663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-786cd4fc-9596-4075-9f43-966a2838ad02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:23.220485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:23.221472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:23.222374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:23.222374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:23.222374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:23.222374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:23.223453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.224611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.225109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.225623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.226571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 379/731] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:23.228407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fdc73-21bc-41bd-92da-9310474ccea8 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.228806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fdc73-21bc-41bd-92da-9310474ccea8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.228972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fdc73-21bc-41bd-92da-9310474ccea8 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.234621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 380/732] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:23.234621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fdc73-21bc-41bd-92da-9310474ccea8 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.244505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.246642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-484e9019-4778-4405-8d6a-dd7dae336bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:23.267232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:23.268644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:23.269535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:23.269535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:23.269535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:23.269535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:23.276193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 353/733] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:23.276193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.276193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.276193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.298867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:23.299973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:23.346286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:23.377674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fdc73-21bc-41bd-92da-9310474ccea8 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.386688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 354/734] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 04 09:05:23.396311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.418760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.419430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.419716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.419990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.420963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:23.421458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.428321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.439558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 381/735] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:23.444420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.501616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:23.529550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4e1a-40c3-4aa3-bd18-183cb092ae7d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.571639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.572762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.573279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.573783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.574757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:23.575283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.576969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.579103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f4ec7-cc37-4b59-836f-0be47f9e68f4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.590533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 355/736] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:23.591698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.629679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.630362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.630634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.630906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.631628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.634373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.703085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67205ff3-c2d6-48a5-a835-e42ead508d01 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.720341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 382/737] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:23 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:23.720341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.721071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:23.799069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.799885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.800160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.800464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.803171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:23.814711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.816203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:23.838172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:23.839247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:23.840209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:23.846000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 356/738] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:23.854668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:23.919098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669302d-e37a-430a-a765-0972a06bf5cb None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:23.961737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:23.963829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.964342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.964813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.976720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:23.978720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-f1fd85dc-6b9b-496a-a42f-e35d5ae880b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:23.994902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:23.995918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:23.997360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 383/739] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:05:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:23.998094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db47b56-c69e-47d9-836c-9b74fb4653d5 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:23.998094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db47b56-c69e-47d9-836c-9b74fb4653d5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:23.998240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db47b56-c69e-47d9-836c-9b74fb4653d5 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:23.999448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 384/740] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:24.005539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.005539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.005539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.009335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6999-6ec2-41e6-abb4-13a782433954 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.025479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:24.026681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:24.053040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.053705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.053981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.054256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.055088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:24.055394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.056857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.077238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 357/741] 10.222.0.46 () {56 vars in 1135 bytes} [Sat Jun 4 09:05:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:24.078217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.097836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.115285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 385/742] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:24.116510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.163258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.219245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702217a5-1b7d-483e-b851-34b6de0c8a6e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.260662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b28ba-b561-4203-9b57-df84bef5ee8a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.262204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.263045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.263335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.263612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.265296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:24.265620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.267633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.289501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 358/743] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:24.290763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.311680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.313030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.314102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.314934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.384708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.397648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.399961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.416668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28ede-9957-443b-a199-40817fa61ce3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.432645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.452609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.453419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.453700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.453975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.466169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:24.474062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:24.475034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:24.475873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 359/744] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:24.477233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.508732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 386/745] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:05:24.509617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.579567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4df60c-9ddb-4d86-ab0d-ec45e8b1b5dd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.604694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.615193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004af56-30d4-4bb1-b105-e3ab596b67b9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.615468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.616076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.616409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.616684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.628550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.630024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:24.648765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:24.650994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:24.653281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:24.653281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:24.653281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:24.653281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:24.653826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.653826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.653826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.653826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.654401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:24.654401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.654912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 360/746] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:24.656407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.656565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.665369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 387/747] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:24.666387 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.683586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.763246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d17b1e-2082-40a9-9ed4-d6d52680ba91 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.772041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbc54f-57d4-468a-b631-51d7d3f9d5c6 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.802973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.804118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.804574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.805094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.809246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.809974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.810326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.810687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.811471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:24.811842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.813519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.818182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.819842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:24.831401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 388/748] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:24.831401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.840516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:24.846368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:24.847243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:24.848129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:24.848129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:24.848129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:24.848129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:24.850129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 361/749] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:24.851086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:24.933178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:24.939995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b57b8-36ae-4fdb-a349-c1227d38502e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.961515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e650f31-24eb-4910-a2f7-6282dcb3664c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:24.977407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:24.978218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:24.978497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:24.978771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:24.990379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:24.991849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.002223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.003389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.003861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.004375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:25.011734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:25.012794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:25.013676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:25.013676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:25.013676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:25.013676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:25.013676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 389/750] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:25.014454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.015425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.017856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:25.036178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:25.037321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:25.038325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:25.044648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 362/751] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:25.045804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.084776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.117879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cd1cf-9042-4f3f-a4c0-79f3f8f76830 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.156112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.156822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.157104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.157415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.158154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:25.158460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.160154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.163995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7cc3-7f71-4f3d-bd64-b95f731cbc7f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.178013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 390/752] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:25.179064 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.209912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.211127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.211711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.212499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.213700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:25.214154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.216738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.220568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.234632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 363/753] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:25.235833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.312820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aec31-aa84-40f9-946e-89f2d4b60b4f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.344103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.352263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cd02b-1fca-431d-9811-ad130d365336 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.359362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.360030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.360362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.360659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.361436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:25.361744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.363762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.382311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 391/754] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:25.383434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.394531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.395620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.396107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.396617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.404102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.405848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:25.422721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:25.424200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:25.425451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:25.427793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 364/755] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:25.428831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.489150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.511200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e65ef3-415d-4a7d-8e9d-ab4b48d089bd None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.536531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93f46-7ed3-449a-ba90-0536421a9860 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.545746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.546554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.546835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.547122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.555008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.556468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:25.572237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:25.573462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:25.574714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:25.577675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bf3d1-87a1-4e7a-81ed-7532a34dac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.578329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 392/756] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:25.578762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bf3d1-87a1-4e7a-81ed-7532a34dac00 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.578899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.579268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bf3d1-87a1-4e7a-81ed-7532a34dac00 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.579642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bf3d1-87a1-4e7a-81ed-7532a34dac00 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/5c1d511f4c6341e09d54bf8709efdf8e/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.582047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bf3d1-87a1-4e7a-81ed-7532a34dac00 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=5c1d511f4c6341e09d54bf8709efdf8e, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.584506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bf3d1-87a1-4e7a-81ed-7532a34dac00 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.621873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.677202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0700b-2561-498c-a394-8af3053e5577 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.712599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 365/757] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:25 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/5c1d511f4c6341e09d54bf8709efdf8e/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:25.713646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.714185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400741b-59bb-49b5-a1c4-2dfcd16cee0f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.714302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.714478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400741b-59bb-49b5-a1c4-2dfcd16cee0f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.714595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.714749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400741b-59bb-49b5-a1c4-2dfcd16cee0f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.714843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.715625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:25.715959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.716395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 366/758] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:25.717094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400741b-59bb-49b5-a1c4-2dfcd16cee0f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.722555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81989ff6-f97e-4187-874b-116710d90281 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.746906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 393/759] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:25.749572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.750134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.750691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.775939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:25.778390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:25.873884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400741b-59bb-49b5-a1c4-2dfcd16cee0f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:25.884957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 394/760] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:25.886199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:25.893509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.918330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:25.919372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:25.919862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:25.920415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:25.921611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:25.922205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:25.924468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:25.940523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:25.946789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 367/761] 10.222.0.46 () {56 vars in 1149 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:25.947662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.020316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565f95-995d-4501-b07d-006168b4a48a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.064930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e97233-efbc-4232-9990-c1ef102b01b5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.069252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.070211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.070703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.071191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.072227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=85a16b32249c48eb89cf0d7cd55e5de7) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:26.073365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.075525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.084155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 395/762] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/role_assignments?user.id=85a16b32249c48eb89cf0d7cd55e5de7&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:26.092495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.132168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.133145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.133540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.133824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.134508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.135958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:26.152884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:26.153912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.154775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 368/763] 10.222.0.46 () {56 vars in 1129 bytes} [Sat Jun 4 09:05:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:26.160649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.223854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:26.296137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:26.306653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce61f3-b945-41b1-b7db-6dc3a40b3f5f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.307889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6260-3152-4842-b393-883cc9dda3f2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.351572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd490a21-e687-4689-be6f-c4db37f5e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.352228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd490a21-e687-4689-be6f-c4db37f5e488 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.352642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd490a21-e687-4689-be6f-c4db37f5e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.352921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd490a21-e687-4689-be6f-c4db37f5e488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.356491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.356491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.356491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.356491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.357180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:26.357180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.358010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598c447-c88b-4b98-995b-9a0e201703a7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.367929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 369/764] 10.222.0.46 () {56 vars in 1124 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:26.369222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d48445-4e53-4f22-9275-5d5e33c59d2a None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.369516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d48445-4e53-4f22-9275-5d5e33c59d2a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.369800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d48445-4e53-4f22-9275-5d5e33c59d2a None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.371093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 370/765] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:26.372048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d48445-4e53-4f22-9275-5d5e33c59d2a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.427791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd490a21-e687-4689-be6f-c4db37f5e488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.429641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd490a21-e687-4689-be6f-c4db37f5e488 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.488891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:26.493865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d48445-4e53-4f22-9275-5d5e33c59d2a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.528328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:26.533987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.534672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.534956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.535233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.535949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:26.536256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.537806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.538716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 396/766] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:05:26.540344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.540804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.541346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.545521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 371/767] 10.222.0.46 () {56 vars in 1124 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:26.546502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.559774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:26.561393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:26.637180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 397/768] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:26.638535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.683102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb82eb3-41d5-485f-ab8e-665d032d937c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.723934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.725104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.725617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.726116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.737537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.739302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.754278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5af124-c97e-47bd-9e6b-114c66466313 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:26.758593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:26.759552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:26.760510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:26.760510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:26.760510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:26.760510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.762722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 372/769] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:26.763742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.784564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:26.792735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:26.795721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.796697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.797217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.797677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.798813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] RBAC: Adding query filter params (interface=public, service_id=212e21b5bb02424b8359ef066473a877, region_id=RegionOne) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:26.799361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.800997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.809465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 398/770] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/endpoints?service_id=212e21b5bb02424b8359ef066473a877&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:26.810623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.897839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba12d-5fcd-4006-baae-4b9db027fab9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.939786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.940533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ae2bb-d77b-4d11-8015-6df41a4a769a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:26.940656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.940898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.941183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] PATH_INFO: `/v3/services/network` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.941846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.943474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:26.956535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:26.957579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:26.958407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:26.962674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 373/771] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:26.963550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:26.981511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:26.982656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:26.983226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:26.983796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:26.984938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:26.986038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:26.987802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.003566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 399/772] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:27.004865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.059108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:27.064239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:27.082018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ff8bc-8e1a-4e2b-9faa-4c490db72919 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.120938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.121876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.122164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.122445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.123179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.123805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dab87a-b5e4-425c-b93a-9db219350e39 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.124609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.138452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 374/773] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:26 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:27.139392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.194151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.194839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.195121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.195399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.196149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] RBAC: Adding query filter params (name=network) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:27.196488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.197918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.210816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 400/774] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:27.216910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.335319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d7412-9dcf-442e-9a23-8e85578b3c5f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.386723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.387554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.387843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.388121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.397693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.399162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.404238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23d902-ac0c-4f60-8cd5-0401f1f66847 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:27.414660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:27.415603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:27.416549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 375/775] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:27.422466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.423434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:27.426787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:27.450316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.451370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.451857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.452400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.453308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] RBAC: Adding query filter params (type=network) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:27.453842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.456000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.462723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 401/776] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:27.469152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.534819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee433ea-2659-4dec-88dd-d0218975bfb4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.570805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.571471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.571770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.572050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.572873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:27.573186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.574633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.578083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e405c-095f-4a75-8f75-a2466f39a882 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.596769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 376/777] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:27.602934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.637784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.638790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.639252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.639714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.640811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] RBAC: Adding query filter params (interface=public, service_id=9f5bb4f2340c49949535cee6eba59a52, region_id=RegionOne) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:27.642832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.644453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.650754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 402/778] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/endpoints?service_id=9f5bb4f2340c49949535cee6eba59a52&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:27.665784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:27.694778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:27.727499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:27.789143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85995b-cc9d-42af-8365-122cecc7e927 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.792488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ececeb-b567-4424-b68b-24863c345c65 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:27.833685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e8522-10b4-47be-9b27-4b421bd49667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.835110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8522-10b4-47be-9b27-4b421bd49667 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.836600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8522-10b4-47be-9b27-4b421bd49667 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.837625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8522-10b4-47be-9b27-4b421bd49667 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.838859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e8522-10b4-47be-9b27-4b421bd49667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.840951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:27.841870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:27.842802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e8522-10b4-47be-9b27-4b421bd49667 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.843300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:27.845026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:27.846143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=d1bb3c381d7c47aab9d79db14debc319) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:27.847706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:27.849808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05302-2a77-43ec-9eb8-14711ff93b80 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:27.853302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 403/779] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:27.860275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 377/780] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:27 2022] GET /identity/v3/role_assignments?user.id=d1bb3c381d7c47aab9d79db14debc319&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:28.295866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e87b2f-9548-46ff-bcc5-b3a857c69137 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:28.296568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e87b2f-9548-46ff-bcc5-b3a857c69137 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:28.297085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e87b2f-9548-46ff-bcc5-b3a857c69137 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:28.299699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 404/781] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:28.306158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8fa8a-869b-43dd-bb1f-446872ff061b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:28.306792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8fa8a-869b-43dd-bb1f-446872ff061b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:28.307275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8fa8a-869b-43dd-bb1f-446872ff061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:28.326339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d1e8fa8a-869b-43dd-bb1f-446872ff061b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:28.327914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d1e8fa8a-869b-43dd-bb1f-446872ff061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:28.419133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 378/782] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:28.456923 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e87b2f-9548-46ff-bcc5-b3a857c69137 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:28.612617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e87b2f-9548-46ff-bcc5-b3a857c69137 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:28.620562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:28.661017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:28.661661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:28.661946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:28.662227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:28.663830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8607b8c-326f-407a-b713-56830ad278b5 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:28.664377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8607b8c-326f-407a-b713-56830ad278b5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:28.664882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8607b8c-326f-407a-b713-56830ad278b5 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:28.666805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 379/783] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:28.672959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:28.673515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:28.674019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:28.676772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:28.688351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:28.689409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:28.690249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:28.691695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 405/784] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:28.692703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:28.715800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:28.725597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:28.808659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c7349-42d3-4b4e-a727-9ccb0842e2ee None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:28.852996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:28.853667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:28.853944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:28.854215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:28.854963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:28.855264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:28.856650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:28.868744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:28.872416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 406/785] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:28.879965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:28.895633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 380/786] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:28.915975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:28.924466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.046905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09bc3c-d108-424b-841f-33ddb0022dac None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.070225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8843-ee1f-4afe-80b1-d2b8067fb3ea None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.087079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.087972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.088362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.088703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.101250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:29.109847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:29.111364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:29.112206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:29.112903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 381/787] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:29.118602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.140552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.141273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.141615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.142437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.154909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:29.156534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:29.176309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:29.177254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:29.178133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:29.178133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:29.178133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:29.178133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:29.178133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 407/788] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:29.183292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.235275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:29.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7d650-351c-4485-97a4-0e22f9c1673e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.360089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:29.382870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5a18-0fbc-441a-bac1-0e5313179e9b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.424233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.425544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.426006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.426510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.428018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:29.428539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:29.430238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:29.443269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 408/789] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:29.445638 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.446852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.447313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.447832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.448878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.450182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:29.451021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:29.461003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:29.479263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 382/790] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:29.483373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.606104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea0a22-2c6c-4618-805a-ccca03e6eb60 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.621716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:29.653608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.654417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.654696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.654969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.680174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:29.681599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:29.685110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdbf92-40d5-42b0-9548-71426adc5d5d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:29.708754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:29.709730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:29.710626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:29.710626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:29.710626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:29.710626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:29.722136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 383/791] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:29.725907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.727027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.727510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.727866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.728232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.747453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:29.748942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:29.755253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:29.771234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:29.772209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:05:29.773174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:29.774339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 409/792] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:29.783512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.899974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb309cc-ef85-4133-98f4-2bd8fd2978f7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:29.936676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:29.937374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:29.937665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:29.937940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:29.938995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:29.939357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:29.940937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:29.964540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 410/793] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:29.965868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:29.990280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966660-b0cf-4063-8eb7-71b51ab1fb1d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.035858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.058565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.059237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.059519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.059797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.060554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:30.060856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.062309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54438bfc-c032-44f1-84c2-9f733e201630 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.080027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 384/794] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:30.081540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccec83-7484-4ff6-969d-f9d6b8cf5bcf None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.081829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccec83-7484-4ff6-969d-f9d6b8cf5bcf None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.082102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccec83-7484-4ff6-969d-f9d6b8cf5bcf None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.083671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 385/795] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:30.086838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0feaffa-7340-4bf3-a127-520f8350d5a7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.088387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccec83-7484-4ff6-969d-f9d6b8cf5bcf None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.117098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.134191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.135136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.135568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.136559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.222063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.223453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.230251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccec83-7484-4ff6-969d-f9d6b8cf5bcf None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.277812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.279033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.279848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.280298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.294844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.297351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.299086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-832eafdc-6175-4cd6-85c3-0eda72bdb30f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:30.318331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:30.319368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:30.320329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:30.324483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 386/796] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:30.326014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52048190-100d-4695-8309-90040ec535c3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.326248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52048190-100d-4695-8309-90040ec535c3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.326719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52048190-100d-4695-8309-90040ec535c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.352603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-52048190-100d-4695-8309-90040ec535c3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:30.353281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 411/797] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 04 09:05:30.354272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-52048190-100d-4695-8309-90040ec535c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:30.354610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.448366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 387/798] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:30.449397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52048190-100d-4695-8309-90040ec535c3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.477808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.491660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0820f7e-6415-452d-bb9f-89dfcf62bf1d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.529212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.529876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.530157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.530429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.531249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:30.531554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.533583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.556394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 412/799] 10.222.0.46 () {56 vars in 1135 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:30.556394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.569625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52048190-100d-4695-8309-90040ec535c3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.572280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.603834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.604964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.605249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.605527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.616421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.617915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-d54d7734-5a06-4029-aa84-49f8f9463069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:30.635800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:30.636920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:30.638004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:30.638004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:30.638004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:30.638004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:30.640447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 388/800] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:30.642939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b8de2-ede6-4832-bdeb-4ac339c8186b None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.643233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b8de2-ede6-4832-bdeb-4ac339c8186b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.643507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b8de2-ede6-4832-bdeb-4ac339c8186b None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.645056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 389/801] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:30.645587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b8de2-ede6-4832-bdeb-4ac339c8186b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.667233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d7cd2-49d4-4f8c-8aeb-b6d6ba7d207b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.705543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.706391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.706718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.706942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.707595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.709006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.718045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:30.722776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:30.723760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:30.724650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 413/802] 10.222.0.46 () {56 vars in 1129 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:30.725510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.760313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b8de2-ede6-4832-bdeb-4ac339c8186b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.794499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.795293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.795578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.795858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.807001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.808489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.808682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-7cb838de-076e-47f6-b68a-bd01266870e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:30.825931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:30.827052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:30.827959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:30.827959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:30.827959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:30.827959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:30.827959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 390/803] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:30.828242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e104-f99b-45ef-b85f-657a9c07f504 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:30.828356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc813e7c-ec4f-4f68-aa84-31023b2de4b3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.828772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc813e7c-ec4f-4f68-aa84-31023b2de4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.828885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc813e7c-ec4f-4f68-aa84-31023b2de4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.846665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dc813e7c-ec4f-4f68-aa84-31023b2de4b3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:30.847833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dc813e7c-ec4f-4f68-aa84-31023b2de4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:30.866759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:30.867419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.867706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.867987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.868737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:30.869048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:30.870450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:30.882665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 414/804] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:30.883473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.915435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 391/805] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:30.916794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ba03-695d-4e68-a0f5-64c203532ca0 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:30.917082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ba03-695d-4e68-a0f5-64c203532ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:30.917357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ba03-695d-4e68-a0f5-64c203532ca0 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:30.918776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 392/806] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:30.921042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ba03-695d-4e68-a0f5-64c203532ca0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:30.973129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:30.989404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80edfd-5b4c-491f-a3e0-59707069df3e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.024588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.025173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.025457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.025735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.026432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.026741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.026946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ba03-695d-4e68-a0f5-64c203532ca0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.028118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.032626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 415/807] 10.222.0.46 () {56 vars in 1124 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.033917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.038872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.062267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.062925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.063203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.063476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.064819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.065132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.066519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20749c-9688-4289-9c23-4dfc22907b82 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.077725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 393/808] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.079021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.079919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.080352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.096489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:31.097687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:31.133964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff728f7b-3a4a-4950-8bc2-0a456d636c13 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.163625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 394/809] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:31.164330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.180179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.181013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.181296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.181575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.189795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.191249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.198018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:31.205501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:31.206439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:31.207436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 416/810] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:31.210291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.269644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9564a-2165-4fc2-a624-2871c8fe1a47 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.270603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.305545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.306621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.306897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.307170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.308393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0da538-6414-402b-8a6f-710b14470f24 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.317961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:31.325772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:31.327215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 395/811] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:31.328214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.348509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.349169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.349445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.349717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.350424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.350718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.352071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.356846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 417/812] 10.222.0.46 () {56 vars in 1124 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.357606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.425727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.431740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69104719-4625-4d19-862d-5a3345847b3b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.458733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e81826-3b29-49fd-8a1a-587b4a72754d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.465720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.466517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.466799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.467072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.473821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.475284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.484648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:31.489420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:31.490375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 396/813] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:31.491444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.494360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.495470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.495470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.495717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.496603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.497977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:31.510377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:31.511584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:31.512466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 418/814] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:31.513271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.597592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184682-6e94-47e6-a971-6375ef3fb6b3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.611767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297bbf6-3121-46bd-bceb-4ddda1557cb1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.631979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.632672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.633012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.633235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.634027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.634338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.635778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.646422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.647089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.647371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.647646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.648334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 397/815] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.648447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.648721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.649152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.650092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.658531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 419/816] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.659346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.669059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.714595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.751811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae336c-eeb6-4cf9-a18e-9295b6e7b56f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.759330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d6a89-02ed-44c1-a9d8-dd756fa75d5b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.786521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.787176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.787462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.787737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.788514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.789886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.792864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.793599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.793885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.794246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.795031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.795401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.796794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.810867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 420/817] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.811738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.909254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321efe2e-b9b9-468b-ac0e-2ad6b56848f2 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:31.919327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 398/818] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:31.924974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:31.938112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:31.956643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:31.957437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:31.957778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:31.958116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:31.958898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:31.959208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:31.961233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:31.971303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 421/819] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:31.972274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.002321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:32.036009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337bdf2-a594-4027-83b5-936286389b9d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.075557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.076503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.081622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.081622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.081622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=5c1d511f4c6341e09d54bf8709efdf8e) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:32.081622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.081622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.088354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 399/820] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/role_assignments?user.id=5c1d511f4c6341e09d54bf8709efdf8e&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:32.088354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.102226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3173d3-a7ad-458d-bb93-149a5c5050c0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.155902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.157122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.157660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.158152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.171163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.174381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:32.195320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:32.196687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:32.197761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:32.197761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:32.197761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:32.197761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:32.198609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 422/821] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:32.199790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.231677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b6e8e-98d5-4eeb-a0ed-e13b08c6839d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.236172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:32.277643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.279110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.279603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.280212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/85a16b32249c48eb89cf0d7cd55e5de7/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.283193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=85a16b32249c48eb89cf0d7cd55e5de7, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.285248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.291943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:32.335856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95c7c-8438-45ab-9223-c6791577327a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.375202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.375879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.376185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.376526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.377247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:32.377553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.379472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.385130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 423/822] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:32.386155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.463967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 400/823] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:32 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/85a16b32249c48eb89cf0d7cd55e5de7/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:32.465296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.514628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:32.534178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144973b-5d24-49a5-bc1f-6d12e81c32b7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.606458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.607402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.607874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.608331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.610559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:32.611574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.613182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.638219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 424/824] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:32.644757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.683584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca40179-917e-4b3d-ac0f-967c368e580c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.755898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.757042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.757328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.757607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.758960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.760544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.783671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e64b-b9d5-4859-bc7d-7cc474525b70 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:32.824436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:32.825252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:32.825535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:32.825869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:32.834996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:32.836481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:32.852817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:32.853839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:32.854769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 425/825] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:05:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:32.862024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:32.877867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:32.906516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 401/826] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:32.988188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7b461-e87f-40e9-b5ba-b3cb0c94e956 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:33.034684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:33.035297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:33.035585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:33.035865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:33.036762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:33.037074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:33.038547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:33.059212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 426/827] 10.222.0.46 () {56 vars in 1149 bytes} [Sat Jun 4 09:05:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:33.065265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:33.132217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:33.215545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc38b8a-b368-4c2b-9dcc-2b42c4b40e1e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:33.257512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:33.258329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:33.258615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:33.258894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:33.272318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:33.276328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:33.291509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:33.292454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:33.293374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:33.293374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:33.293374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:33.293374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:33.293374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 402/828] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:33.300012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:33.364651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:33.402805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277a9e1-a2fa-4134-975e-e409bf23c57e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:33.439948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:33.440650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:33.440928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:33.441206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:33.441985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:33.442299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:33.443668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:33.455974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 427/829] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:33.461271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:33.580216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c747db1-5d98-4f91-a0d2-3b80378a589e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:33.582576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:33.622467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:33.623266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:33.623544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:33.623817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:33.633416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:33.634897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:33.651085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:33.652048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:33.653064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 403/830] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:33.661562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:33.764060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ba578-2bee-497b-8f62-355176914e42 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:33.788397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:33.803387 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:33.804049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:33.804364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:33.804650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:33.805432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:33.805742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:33.807236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eedaa0-fbc8-41b1-b41f-8726413f52dd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:33.820320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 428/831] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:33.830792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:33.944576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4300939-dfa7-476a-a9d8-e013fadafcf1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:33.981567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ffcdd-5d9b-45ec-83da-22a9e7570052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:33.982530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ffcdd-5d9b-45ec-83da-22a9e7570052 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:33.982814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ffcdd-5d9b-45ec-83da-22a9e7570052 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:33.983136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ffcdd-5d9b-45ec-83da-22a9e7570052 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/d1bb3c381d7c47aab9d79db14debc319/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:33.999373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ffcdd-5d9b-45ec-83da-22a9e7570052 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=d1bb3c381d7c47aab9d79db14debc319, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:34.000869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ffcdd-5d9b-45ec-83da-22a9e7570052 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:34.081385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 404/832] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:33 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/d1bb3c381d7c47aab9d79db14debc319/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:05:34.378742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a4d98-abcd-46db-935f-b2b4e0ae752e None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.378926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a4d98-abcd-46db-935f-b2b4e0ae752e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.379320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a4d98-abcd-46db-935f-b2b4e0ae752e None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.381475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 429/833] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:34.387233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.387893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.388430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.407580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:34.409175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:34.420235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:34.481608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 405/834] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:34.504988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a4d98-abcd-46db-935f-b2b4e0ae752e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:34.611849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a4d98-abcd-46db-935f-b2b4e0ae752e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:34.648634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:34.649439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.649720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.649995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.650647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:34.652129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-0f893492-b813-4abd-977f-5648d2be3e85 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:34.664667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:34.665728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:34.666757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 430/835] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:34.672495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:34.735534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91dfa8-c944-407a-a126-552f62669224 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.736568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91dfa8-c944-407a-a126-552f62669224 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.736922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91dfa8-c944-407a-a126-552f62669224 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.738591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 431/836] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:34.743554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.743911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.744263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.762491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:34.763805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:34.783422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0daf20-4501-4c4c-bfe1-243f28502680 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:34.816701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:34.822055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:34.822978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.823447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.823949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.824882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:34.825387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:34.826969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:34.832854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 406/837] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:34.844363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 432/838] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:34.844559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:34.854043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:34.860200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:34.949041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c90a1-913b-42d5-8dca-11a1096f940e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:34.984759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:34.985742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:34.986681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:34.987112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:34.987399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41cdbc-cc66-4c16-bfd7-16e3eee0135b None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:34.988117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:34.988599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:34.990135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09c326-e2ed-4923-96e2-59d5a231cffc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:34.995690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 407/839] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:35.018476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a6bc4-5ae2-4e1e-b7d9-4c48e2f375e7 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.018794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a6bc4-5ae2-4e1e-b7d9-4c48e2f375e7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.019029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a6bc4-5ae2-4e1e-b7d9-4c48e2f375e7 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.024747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 408/840] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:35.026558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.026993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.027347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.028572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.029402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.029689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.029969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.030634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:35.032044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:35.046205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:35.047558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:35.048839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:35.049728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:35.050550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 433/841] 10.222.0.46 () {56 vars in 1132 bytes} [Sat Jun 4 09:05:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:35.056490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.105728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:35.132421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 409/842] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:35.160115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.162593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:35.191087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c55e3-b326-438e-b04c-be26be49c18c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:35.234795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.235884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.236205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.236507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.237222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:35.237529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:35.238906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:35.244907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 434/843] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:35.250739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.283120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f248d-6572-4008-bf1e-a31888eee6c5 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:35.324209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.325106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.325389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.325663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.341209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:35.349769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:35.350724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:35.351552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 410/844] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:35.360329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.390593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952f8f5-4608-47bf-bcc1-17ee651228ab None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:35.398392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:35.426457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.427130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.427423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.427695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.428454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:35.428765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:35.429472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:35.430248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:35.440325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 435/845] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:35.481049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256db13-6954-40ca-a7a2-433c9bb7cdbc None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:35.520124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.520749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.521029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.521305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.522060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:35.522363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:35.523773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:35.534331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 411/846] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:35.539709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.655182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a052711-cf0d-4566-b561-c16a91b8a965 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:35.690290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.691081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.691365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.691642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.702564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:35.704017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:35.722940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:35.723868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:35.724768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:35.724768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:35.724768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:35.724768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:35.728518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 436/847] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:35.733391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.836558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63624a3a-0861-4c7f-a243-62b8fadd14f4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:35.865626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:35.879119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cef855-1565-4289-999a-4515880aa699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:35.879730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cef855-1565-4289-999a-4515880aa699 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:35.879821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cef855-1565-4289-999a-4515880aa699 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:35.880311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cef855-1565-4289-999a-4515880aa699 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:35.880993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cef855-1565-4289-999a-4515880aa699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:35.881300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cef855-1565-4289-999a-4515880aa699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:35.887110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cef855-1565-4289-999a-4515880aa699 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:35.896210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 412/848] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:35.902069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:35.928708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:36.016134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9c179-92fb-4973-a903-af99e31dba44 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.051620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.052781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.053242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.053708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.062651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:36.065021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:36.079798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:36.080795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 04 09:05:36.081751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:36.084044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 437/849] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:05:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:36.089520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cef855-1565-4289-999a-4515880aa699 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.197984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cef855-1565-4289-999a-4515880aa699 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.235517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.236263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.236579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.236856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.237714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:36.238024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:36.239512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e005b1-f752-4f61-9706-f87a960480df None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:36.258654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 413/850] 10.222.0.46 () {56 vars in 1135 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:36.264212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.352315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:36.370731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedaf15-6928-4c6d-9b8e-8bf203c19e9e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.406614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.407262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.407573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.407850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.464396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c59e0b-d3d6-42b5-9559-b30f43017775 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.464684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c59e0b-d3d6-42b5-9559-b30f43017775 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.464957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c59e0b-d3d6-42b5-9559-b30f43017775 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.467001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 414/851] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:36.477439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.477932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.478289 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.483400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:36.484953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:36.499530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:36.502123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:36.552643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:36.573597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:36.601622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 438/852] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:05:36.602078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 415/853] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:36.603036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.627661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.734024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81465204-7cc8-4eda-85ea-bf810c6a48a7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.738125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806df23-6301-41bd-bf20-dec240abad2e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.773551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.774187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.774543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.774888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.783050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.784623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.785379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.785822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.786168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:36.787713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:36.799893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:36.806955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:36.808039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:36.809191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:36.809191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:36.809191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:36.809191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:36.810686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:36.811595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:05:36.812488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:36.815370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 416/855] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:36.815735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 439/855] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:36.817787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.821600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.857798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:36.871007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:36.934855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03892f04-3e63-42d6-b7e1-64e9f7c15aa4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.947109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d587-0fc6-4b9f-8fde-a19069f8631c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:36.973000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.973458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.973744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.974022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.974767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:36.975076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:36.976541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:36.990423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:36.991365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 440/856] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:36.991714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:36.992203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:36.992700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:36.992807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:36.993858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:36.994364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:36.996199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.017342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 417/857] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:37.018475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.108159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:37.114906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177d66-551d-4d93-9d5a-19de91bc8902 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.148869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:37.152046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.153261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.153532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.153908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.160539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17504fd7-d6e1-4392-9fac-9d24406c32c0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.168326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.170445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:37.195968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:37.197053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:37.198078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:37.198078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:37.198078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:37.198078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:37.200312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 441/858] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:37.205154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.205797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.206679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.207174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.207656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.217693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.220879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:37.241459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:37.242446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:37.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:37.250803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 418/859] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:37.251983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.341932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab2b5-e99c-48a0-8dda-465d2433bf69 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.365483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08072f9a-d85b-4adf-a65f-7ddb85508e22 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.386480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.387414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.387867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.388209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.389220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:37.389453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.390937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163bb7dc-a534-48a5-ae8e-ae976a127670 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.405847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 442/860] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:37.407682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d12ec8-cb34-4940-aa3a-76eddd14b6b7 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.409076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d12ec8-cb34-4940-aa3a-76eddd14b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.409357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d12ec8-cb34-4940-aa3a-76eddd14b6b7 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.412249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 443/861] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:37.412249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d12ec8-cb34-4940-aa3a-76eddd14b6b7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.415710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.416537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.416871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.417149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.417942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:37.418249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.419700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6800e-56dc-406e-bab8-30a20255146a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.424105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:37.432731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 419/862] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:37.438008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.438008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.438008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.450970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:37.452558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:37.465314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:37.533204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d12ec8-cb34-4940-aa3a-76eddd14b6b7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.535999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 420/863] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:37.545184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.572408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.573309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.573598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.573882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.581665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.583109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:37.599016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:37.600161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:37.601157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 444/864] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:37.601754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.664643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fad9a-961a-4be4-85d3-689f1e4f54ec None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.673397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:37.703840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.705119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.705416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.705674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.706503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=85a16b32249c48eb89cf0d7cd55e5de7) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:37.706805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.707581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5d7ab-610c-400d-81c0-f749a302b45e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.708246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.724316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 421/865] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/role_assignments?user.id=85a16b32249c48eb89cf0d7cd55e5de7&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:37.724316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.733840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:37.757616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b85f3-4655-436e-91fb-db21a577e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.758530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b85f3-4655-436e-91fb-db21a577e683 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.759005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b85f3-4655-436e-91fb-db21a577e683 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.759380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b85f3-4655-436e-91fb-db21a577e683 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.760326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b85f3-4655-436e-91fb-db21a577e683 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.761848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b85f3-4655-436e-91fb-db21a577e683 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.841955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087da99-55f2-449c-9510-74e3a52a628f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:37.860756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 445/866] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:37 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:37.862432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11212e-7c75-49c1-a407-a0d632296e83 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.863019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11212e-7c75-49c1-a407-a0d632296e83 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.863379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11212e-7c75-49c1-a407-a0d632296e83 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.865108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 446/867] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:37.871299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.871299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.919957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.919957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:37.919957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:37.920179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:37.920179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:37.920179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:37.920179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:37.920836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:37.920836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:37.920836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:37.941877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 422/868] 10.222.0.46 () {56 vars in 1149 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:37.943830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:37.990803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 447/869] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:38.016015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:38.019817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:38.069044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bfef8-fe18-4d76-8473-2bad15f15ecb None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:38.094879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:38.116028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:38.116268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:38.116560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:38.116840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:38.150685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be85cb-e28e-4468-a58b-d842efaa8587 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:38.192228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:38.193760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:38.194581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:38.195043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:38.196243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:38.198453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:38.216317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:38.217812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:38.285344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:38.311425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 448/870] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:38 2022] POST /identity/v3/services => generated 269 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:38.347417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 423/871] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:37 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:05:38.351054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:38.475634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4e2a-8092-46a9-acfe-9ba7881f19a9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:38.492042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:38.515572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:38.516422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:38.516704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:38.517012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:38.533269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:38.535145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:38.555080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:38.556027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:38.557173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:38.557173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:38.557173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:38.557173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:38.557173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 449/872] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:38.563705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:38.675008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17114c-d3ce-4490-acc4-fe3f9907078a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:38.711505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:38.712576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:38.713107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:38.713580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:38.714524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:38.715030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:38.716648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:38.733977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 424/873] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:38.735597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:38.843000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe40d6-6c71-4ec2-8f28-dc33588c01c1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:38.879565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:38.880388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:38.880673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:38.880949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:38.887892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:38.889366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:38.903656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:38.904688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:38.906024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 450/874] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:38.913181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:38.939308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:39.028739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38c85b-1152-45dc-b4f0-3c9d0a9ec5f3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:39.063487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:39.064219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:39.064578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:39.064856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:39.065678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:39.065992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:39.067538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369b3fa-20cb-4f20-8a9a-f89b5b5b12dd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:39.082234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 425/875] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:39.088519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:39.118978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:39.199586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed183b-f5a3-4086-970b-a52dd4c309c8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:39.235945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:39.236643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:39.236929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:39.237210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:39.238048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=d1bb3c381d7c47aab9d79db14debc319) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:39.238364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:39.239807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a14f2-c2d4-49ad-9198-bcdfdb6b47fa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:39.246961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 451/876] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:39 2022] GET /identity/v3/role_assignments?user.id=d1bb3c381d7c47aab9d79db14debc319&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:40.145979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c295-972c-4465-becc-2b6f39d86cb3 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.146506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c295-972c-4465-becc-2b6f39d86cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.146506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c295-972c-4465-becc-2b6f39d86cb3 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.148158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 426/877] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:40.153560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.153977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.154149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.172079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:40.173886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:40.204473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2d9f1-85a2-4bfb-bf22-ecb76926aee4 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.204800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2d9f1-85a2-4bfb-bf22-ecb76926aee4 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.205037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2d9f1-85a2-4bfb-bf22-ecb76926aee4 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.206628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 427/878] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:40.211904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.212181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.212482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.229835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:40.231136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:40.246043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 452/879] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:40.270837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.293595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:40.312893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 428/880] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:40.335251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.374768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afeef3-d606-4bdc-9e67-2f235c377878 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:40.384698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:40.414630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:40.414630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.415022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.415456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.416256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:40.417877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:40.430636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:40.431915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:40.432810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:40.440392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 453/881] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:40.442195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.442340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cd111-398a-4fe2-9619-15e0cbefcb19 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:40.482568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:40.483515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.483883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.484229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.485018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:40.486585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:40.500193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:40.501301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:40.502202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:40.506555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 429/882] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:40.507477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.556866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6d6be-7017-4c2a-8e0d-13a1a6ddc9f8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:40.568726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:40.591851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:40.594381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:40.595882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.596445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.596931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.597896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:40.598447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:40.600053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:40.610134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 454/883] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:40.611566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.617266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f62cd-a622-4560-bb1c-161b070e7e95 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:40.653786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:40.654542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.654873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.655155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.655862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:40.656166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:40.658241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:40.668215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 430/884] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:40.669200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.725968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69b284-6a27-47af-aa83-17ac96b81041 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:40.766014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:40.766616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.769456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.769456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.769456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:40.769456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:40.770423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7818-077f-4ad0-968e-527ab525ba42 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:40.780789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 455/885] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:40.793389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612865e-6e8d-4d5c-a0ea-bfe76bb15d03 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.793718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612865e-6e8d-4d5c-a0ea-bfe76bb15d03 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.793895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff65b5-9f8d-40f9-8308-b77170368a45 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:40.793997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612865e-6e8d-4d5c-a0ea-bfe76bb15d03 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.799942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 456/886] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:40.801470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.801746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.802021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.820762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:40.822478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:40.831716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:40.844200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:40.845335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:40.845828 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:40.846969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:40.848329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:40.848890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:40.851418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:40.859300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 431/887] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:40.865043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:40.869851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:40.908609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 457/888] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:40.928890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.001198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd53d7f-f76b-4a45-92e3-4697c2a383b5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.040235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.041056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.041344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.041622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.042422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] RBAC: Adding query filter params (interface=public, service_id=45ff293a2d8244948485e04d31a57962, region_id=RegionOne) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:41.042735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.043685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d98e7-7acd-4157-bbcf-a0b3e76188ec None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.044128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.052521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 432/889] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/endpoints?service_id=45ff293a2d8244948485e04d31a57962&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:41.061372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.084269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.085333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.085622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.085905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.086757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.088113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:41.102046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:41.103056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:41.103900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:41.107666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 458/890] 10.222.0.46 () {56 vars in 1132 bytes} [Sat Jun 4 09:05:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:41.112016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.128654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:41.149928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:41.186982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae3550-879b-4f72-ba41-c04562da38d0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.228884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.230475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.231227 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.231692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.232850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.235081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.242632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0f70d-ee11-42c7-854d-9c9ae44d56fe None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.253303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 433/891] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:41 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:05:41.289756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.290494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.290783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.291066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.291795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:41.292113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.293561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.300215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 459/892] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:41.306134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.428318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30629f-3676-475c-8fdc-a3947b7162b6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.464921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:41.471457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.472498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.473224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.473751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.474677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:41.475187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.476869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.483064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 434/893] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:41.490098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.568972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:41.601895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51631dbd-b9a2-4997-b9bb-6107f9b2670c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.638271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.638942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.639225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.639501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.640378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] RBAC: Adding query filter params (interface=public, service_id=7d9cf0ea95c34e33841a77ea615cf611, region_id=RegionOne) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:41.640692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.642172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effca96c-cb58-4c75-a711-3fa1abb17261 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.648394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 460/894] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:05:41 2022] GET /identity/v3/endpoints?service_id=7d9cf0ea95c34e33841a77ea615cf611&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:41.653452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.696019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c90c9-1867-4828-a081-da650a1290b7 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.696669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c90c9-1867-4828-a081-da650a1290b7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.697286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c90c9-1867-4828-a081-da650a1290b7 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.699295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 461/895] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:41.706371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.707097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.707579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.730463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:41.731605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:41.767778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12ba28-e50f-4722-97bf-2e6f28b0494c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.804748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.805866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.806322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.811634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 462/896] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:41.811770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:41.811770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:41.811770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:41.819657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 435/897] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:41 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:05:41.832022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:41.900864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:41.953827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2466b-576a-4187-bf2f-d4b0f87cbc0f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:41.994435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:41.995185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:41.995463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:41.995736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:42.008320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:42.016748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:42.018310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:42.019178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 463/898] 10.222.0.46 () {56 vars in 1120 bytes} [Sat Jun 4 09:05:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:42.027485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:42.097906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:42.139768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d500f-34f0-4a13-a939-6827a21587dd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:42.175388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:42.176036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:42.176335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:42.176706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:42.177359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:42.177660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:42.179095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:42.187788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 436/899] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:05:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:42.193577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:42.305795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689ec7-992a-4446-a6fc-7f7a230bf438 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:42.330937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:42.341966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:42.344812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:42.344812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:42.344812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:42.354399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:42.355852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:42.373468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:42.374389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:42.375351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:42.375351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:42.375351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:42.375351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:42.375351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 464/900] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:42.381408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:42.487012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c7d9c-fb93-4bbb-8850-010f3af46ca9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:42.508982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:42.530930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:42.530930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:42.530930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:42.530930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:42.531689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:42.531689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:42.531689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:42.540733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 437/901] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:42.546268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:42.653342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a3a6-5c66-4af8-bcfc-8a2ddedcf3aa None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:42.688581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:42.689975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:42.690254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:42.690525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:42.698548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:42.700000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:42.715031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:42.717291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:42.719484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 465/902] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:05:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:42.724309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:42.835446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f676587-4788-4656-80bc-da5d2212cbd2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:42.872782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:42.873459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:42.873746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:42.874025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:42.874859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:42.875163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:42.877155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d5ad2-8862-404b-969f-f5331c9c5256 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:42.895517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 438/903] 10.222.0.46 () {56 vars in 1149 bytes} [Sat Jun 4 09:05:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:42.901352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:42.940704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.011957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631b51-9417-41d3-a19f-adfe4eb42b0e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:43.046634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:43.047277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.047553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.047828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.120065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:43.121475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:43.155690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.163939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72e55-ebd3-4a2b-b097-75eb59325119 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.163939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72e55-ebd3-4a2b-b097-75eb59325119 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.163939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72e55-ebd3-4a2b-b097-75eb59325119 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.170782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 439/904] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:43.170782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.170782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.170782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.188938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:43.190989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:43.223260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.233831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 466/905] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:05:43.240446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:43.281542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 440/906] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:43.306054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:43.363656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96aa8f-5a55-4364-8b67-fff6a8fb305d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:43.407591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:43.408747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.409171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.409577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.414522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.423843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:43.425575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:43.444823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:43.445966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:43.447099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:43.447099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:43.447099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:43.447099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:43.448907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596e54b-0838-4360-8ab3-b708acda7b43 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:43.453044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 467/907] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:43.455310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:43.493269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:43.494107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.494392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.494673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.496202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:43.497894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:43.565649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b0890-422a-452c-b5f4-cb139b4e28c0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:43.596705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 441/908] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:43 2022] POST /identity/v3/services => generated 241 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:05:43.615041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:43.616052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.616364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.616648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.617403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:43.617738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:43.619247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a71c-01eb-4ee1-b228-ec6f7fb5586c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:43.634307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 468/909] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:43.640202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:43.675052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.735863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9a62-d1de-4ae0-8575-5fd8babbfd86 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.736192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9a62-d1de-4ae0-8575-5fd8babbfd86 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.736467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9a62-d1de-4ae0-8575-5fd8babbfd86 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.738111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 469/910] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:43.743639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.743956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.744194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.761616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ce6b8-927c-41e4-ba5d-17ea31a72f7b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:43.765200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:43.766852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:43.804625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:43.805949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:43.806413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:43.806849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:43.815010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.819926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:43.822226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:43.840663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:43.841792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:43.842990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:43.843444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 442/911] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:43.848893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:43.858366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 470/912] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:43.882212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:43.961659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:43.982380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381654bb-5388-4bfa-a4b2-d69aa405b1db None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:43.993756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc867-936a-4bc1-9e07-00002d6e3fe4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.032204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.033290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.033824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.034294 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.034508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.035146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.035428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.035699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.036193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:44.036481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.036945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.037849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.038838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:44.050433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:44.051505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 04 09:05:44.052397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:44.056232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 443/914] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:44.056525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 471/914] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:44.057793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.061627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.109832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:44.177892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e8bcc-eed4-432d-a7b0-2278b88ad7f9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.196876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca009967-0dad-415a-962d-41ad9b8da887 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.217428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.218400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.218863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.219320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.220411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] RBAC: Adding query filter params (role.id=d43e64e54183445db95c7e057d9b7721, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=d1bb3c381d7c47aab9d79db14debc319) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:44.220905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.222529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.233408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 472/915] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:44 2022] GET /identity/v3/role_assignments?user.id=d1bb3c381d7c47aab9d79db14debc319&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=d43e64e54183445db95c7e057d9b7721 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:44.238140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.238841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.239133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.239414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.240146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:44.240490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.241914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ee7a7-fc72-41b9-918e-c3fb65cd9ec3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.248484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 444/916] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:44.255047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.318391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebe829-3aa6-4c4a-9a46-3096a770d037 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.318730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebe829-3aa6-4c4a-9a46-3096a770d037 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.318962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebe829-3aa6-4c4a-9a46-3096a770d037 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.324514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 445/917] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:44.326418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.326782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.327070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.347022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:44.348317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:44.357743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:44.377987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b265a13-61d6-4b0f-8f89-0b80c872bd87 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.392679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:44.427948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.428665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.428946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.429224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.429941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:44.430291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.431880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.432042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 446/918] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:44.438002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 473/919] 10.222.0.46 () {56 vars in 1114 bytes} [Sat Jun 4 09:05:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:44.449407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.453603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.564169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64eaa54-2409-4988-a6d8-b3cf0fb5ad2b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.577882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4e828-d885-40bc-8c2b-ed17f04ea62c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.601327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.602400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.602774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.603147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.603888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.605540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:44.619925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:44.621151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 04 09:05:44.622482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:44.624554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 474/920] 10.222.0.46 () {56 vars in 1132 bytes} [Sat Jun 4 09:05:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:44.630208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.630540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.631421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.631886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.632377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.633667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.636355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.663636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:44.730644 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364e16-291f-42d4-b954-12b396c9427f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.764258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.764932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.765212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.765482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.766174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:44.766474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.767867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.772820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 475/921] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:44.778071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:44.792470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:44.801725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 447/922] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:44.891542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:44.903524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440692f0-54df-4db2-8a7d-db15dc7e64d9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:44.950313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:44.950994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:44.951276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:44.951556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:44.952270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:44.952674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:44.954080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb12e9-0774-439b-a494-e1d214131b36 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:44.960156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 476/923] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:44.967853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:45.086723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315cac-7d81-4a23-bdee-1b26f6915925 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:45.122029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731544f4-fe8d-4787-af98-e7180e29f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:45.122688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731544f4-fe8d-4787-af98-e7180e29f55d None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:45.122971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731544f4-fe8d-4787-af98-e7180e29f55d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:45.123248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731544f4-fe8d-4787-af98-e7180e29f55d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:45.123998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731544f4-fe8d-4787-af98-e7180e29f55d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:45.125411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731544f4-fe8d-4787-af98-e7180e29f55d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:45.261238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 448/924] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:45.978448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274624-5621-4e93-9eb6-cdb70607e79d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:45.979487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274624-5621-4e93-9eb6-cdb70607e79d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:45.979987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274624-5621-4e93-9eb6-cdb70607e79d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:45.981993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 477/925] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:45.987427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:45.987717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:45.987995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.005492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:46.006677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:46.060570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:46.085882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 449/926] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:46.107944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f274624-5621-4e93-9eb6-cdb70607e79d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:46.218570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f274624-5621-4e93-9eb6-cdb70607e79d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:46.253616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:46.254323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.254605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.254884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.255535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:46.256954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:46.269820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:46.271001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:46.271834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 478/927] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:46.280214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:46.388049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627613ac-cbe9-4b81-aeb0-2c21b7144393 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:46.426659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:46.427329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.427611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.427887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.428633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:46.428940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:46.430325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:46.435603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 450/928] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:46.441820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:46.463487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:46.551937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268b01-19ec-48b9-b059-ca63ca5937ab None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:46.587013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:46.587683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.587961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.588229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.589018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:46.589321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:46.590718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de772240-d5e1-4f04-bb16-d3d51bb1154b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:46.596636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 479/929] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:46.602377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:46.653476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:46.711553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de0328-19eb-4738-a21d-335a6c4880fa None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:46.714091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054171d-11ef-4206-9836-625880c0b101 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.714382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054171d-11ef-4206-9836-625880c0b101 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.714663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054171d-11ef-4206-9836-625880c0b101 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.716268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 480/930] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:46.721625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.721901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.722178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.739925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:46.741162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:46.752507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:46.753294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.753577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.753848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.754640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] RBAC: Adding query filter params (interface=public, service_id=728b1aa4b9ea4c9885ccd4c88424119a, region_id=RegionOne) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:46.754949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:46.756464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:46.762517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 451/931] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:05:46 2022] GET /identity/v3/endpoints?service_id=728b1aa4b9ea4c9885ccd4c88424119a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:46.773299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:46.820488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 481/932] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:46.837150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:46.852873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:46.894659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc133f-2a79-4fa0-87f3-0779deb7c987 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:46.908929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:46.938111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:46.939213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.939674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.940127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:46.941018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:46.942758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:46.956322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb875e6-8b9b-4863-8299-4e85f26038ff None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:46.956734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 452/933] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:46 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:05:46.994342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:46.995171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:46.995448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:46.995721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.008186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:47.016621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:47.017690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:47.018513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 482/934] 10.222.0.46 () {56 vars in 1120 bytes} [Sat Jun 4 09:05:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:47.024267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.143249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36109db5-3a23-4359-8138-dae4e04f917e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:47.181330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:47.182006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.182285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.182560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.183300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:47.183602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:47.185008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e61ff-5c7c-473d-9150-0adc0063dbaa None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:47.194080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 453/935] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:47.199861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.245697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a96d2-f829-4887-ac98-2c95595f9368 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.246003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a96d2-f829-4887-ac98-2c95595f9368 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.246268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a96d2-f829-4887-ac98-2c95595f9368 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.248148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 454/936] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:47.253307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.253604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.253883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.278716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:47.280358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:47.296126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:47.330997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4074-e8e6-49fa-8428-fa8d2de42117 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:47.332739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:47.357324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 455/937] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:47.372309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:47.372309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.372309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.372309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.380963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.392754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:47.394618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:47.413448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:47.414403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:47.415296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:47.415296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:47.415296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:47.415296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:47.417615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 483/938] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:47.423412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.506687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459f3b7-6312-4027-96f0-5fcba3596a8d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:47.539030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accda77-f2d7-4f1b-b259-559db52f08a0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:47.558574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:47.560163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.560698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.561475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.562707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:47.563738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:47.566950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:47.569177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING keystone.server.flask.common [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 04 09:05:47.575293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:47.578894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.578894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.578894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.578894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:47.578894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:47.578894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:47.596318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 484/939] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:47.598478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.628734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.sql.core [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 04 09:05:47.628734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 04 09:05:47.628734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [parameters: {'id': '445c727f1ec04df99f96f96babae5a3f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 04 09:05:47.628734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-0684230f-2a25-4320-9c9f-bca0eb781c13 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 445c727f1ec04df99f96f96babae5a3f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 445c727f1ec04df99f96f96babae5a3f. Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result = self._query(query) Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application conn.query(q) Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 04 09:05:47.646672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result.read() Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 04 09:05:47.647602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application next(self.gen) Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application yield resource Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application next(self.gen) Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.session.rollback() Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.force_reraise() Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise self.value Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application session.commit() Jun 04 09:05:47.648596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._prepare_impl() Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.session.flush() Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._flush(objects) Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application compat.raise_( Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application flush_context.execute() Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 04 09:05:47.649506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rec.execute(self) Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 04 09:05:47.650423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result = self._query(query) Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application conn.query(q) Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result.read() Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 04 09:05:47.651262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [parameters: {'id': '445c727f1ec04df99f96f96babae5a3f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.654922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:47.656955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 445c727f1ec04df99f96f96babae5a3f. Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 456/940] 10.222.0.46 () {60 vars in 1140 bytes} [Sat Jun 4 09:05:47 2022] POST /identity/v3/projects => generated 262 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee578fe-ec6b-4168-bb08-917259b74ef9 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.660103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee578fe-ec6b-4168-bb08-917259b74ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.662053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee578fe-ec6b-4168-bb08-917259b74ef9 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.662053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 457/941] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:47.662053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee578fe-ec6b-4168-bb08-917259b74ef9 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.679443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:47.717308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27bed-ef14-46f3-a852-d28222dc6b9e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:47.756161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:47.757105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.757395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.757674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.765219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:47.766668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:47.769619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee578fe-ec6b-4168-bb08-917259b74ef9 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-0fec0c3c-342f-4681-9611-0e59afffa743 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:47.782708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:47.783930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 485/942] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.789586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.790318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.805560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:47.806749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:47.814981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:47.815981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:47.816477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:47.816919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:47.824729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:47.826615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:47.842489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:47.843397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:47.844330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:47.846267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 458/943] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:47.847216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.848051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:47.880203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 486/944] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:47.881350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:47.943811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:47.963505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e024b8-8ebf-49ee-b78b-2b757394567a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.007371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2c00b-15c7-4f2a-815b-d09a995b0c50 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.008803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.009483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.010507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.010507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.011022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:48.011310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.012792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.032366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 459/945] 10.222.0.46 () {56 vars in 1149 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:48.033614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.052006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.052946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.053620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.053907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.054667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:48.054980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.056966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.088321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 487/946] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:48.088321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.131898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:48.174510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e71de-aaf6-4377-9426-9a8d42a9ac40 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.221273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749b4cc-1bb7-4172-a9ec-f831fae7ad60 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.229611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.230903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.231026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.231602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.232458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.235740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:48.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:48.260213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:48.262070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 460/947] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:48.266468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.272337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:48.300709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.302311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.302611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.302921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.314425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.316001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:48.337553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:48.338505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:48.339395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:48.339395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:48.339395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:48.339395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:48.339395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 488/948] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:48.342644 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.396658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec1970-9576-4930-82e3-94b61066a475 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.444882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.445536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.445816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.446088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.446803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:48.447127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.449168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.456999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 461/949] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:48.459054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:48.469273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.478051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01e88f-1e8d-41b2-b04d-a7b121344206 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.514309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.515057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.515392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.515676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.516435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:48.516745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.518187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.535220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 489/950] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:48.536562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.557079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:48.597931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fc749-1b50-454e-9884-d96e5c6aa107 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.637943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.639086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.639494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.639921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.641061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:48.641581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.643724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.654811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 462/951] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:48.669749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a54a0-3a42-47f9-9578-4b07022f9c27 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.708883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.709694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.709975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.710251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.718199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.719721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:48.736534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:48.737521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:48.738566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:48.742710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 490/952] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:48.744150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.859671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870d56-a28d-404c-9fb0-18c86efd9841 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:48.896787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:48.900673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:48.901353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:48.901641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:48.901918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:48.902724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:48.903093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:48.904646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a871c-9790-4970-b325-8a85a9d90b3c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:48.920466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 463/953] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:48.925550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:48.935944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:49.038651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0ee1-93b2-48d9-a3da-965924f3e31d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:49.077076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca503-3186-4283-baf3-67c316896ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:49.077977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca503-3186-4283-baf3-67c316896ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:49.078261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca503-3186-4283-baf3-67c316896ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:49.078537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca503-3186-4283-baf3-67c316896ad2 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/d1bb3c381d7c47aab9d79db14debc319/roles/d43e64e54183445db95c7e057d9b7721` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:49.081429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca503-3186-4283-baf3-67c316896ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=d1bb3c381d7c47aab9d79db14debc319, role_id=d43e64e54183445db95c7e057d9b7721)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:49.082859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca503-3186-4283-baf3-67c316896ad2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:49.199170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 491/954] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:05:48 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/d1bb3c381d7c47aab9d79db14debc319/roles/d43e64e54183445db95c7e057d9b7721 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:05:49.370138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc903c87-88cd-4e6e-a5b9-69040eeb4b11 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:49.370349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc903c87-88cd-4e6e-a5b9-69040eeb4b11 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:49.370612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc903c87-88cd-4e6e-a5b9-69040eeb4b11 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:49.372580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 464/955] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:49.379098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:49.379425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:49.379646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:49.398445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:49.399666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:49.469993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:49.478876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 492/956] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:49.502043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc903c87-88cd-4e6e-a5b9-69040eeb4b11 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:49.611793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc903c87-88cd-4e6e-a5b9-69040eeb4b11 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:49.649200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:49.650059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:49.650345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:49.650660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:49.651317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:49.652791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:49.665856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:49.666938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:49.667795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 465/957] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:49.675180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:49.781333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1966e-8745-462b-add8-6b475d93d4c1 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:49.818145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:49.819006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:49.819288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:49.819568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:49.820275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:49.820605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:49.822077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:49.827492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 493/958] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:49.837343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:49.893899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:49.948267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a25271-ac61-4908-9b92-2a62e79c9ce4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:49.984541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:49.985594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:49.986128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:49.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:49.987541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:49.988053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:49.989695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d606f4-3f95-4f55-b3cc-cf6211ad4bae None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:49.996562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 466/959] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:50.004481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:50.085603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:50.129332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172163ff-e7a3-401c-b685-0f5b04884bb9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:50.167275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e000d-9382-412b-b82a-e0d0fdc144aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:50.167940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e000d-9382-412b-b82a-e0d0fdc144aa None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:50.168222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e000d-9382-412b-b82a-e0d0fdc144aa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:50.168525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e000d-9382-412b-b82a-e0d0fdc144aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:50.169271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e000d-9382-412b-b82a-e0d0fdc144aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:50.170655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e000d-9382-412b-b82a-e0d0fdc144aa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:50.256819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 494/960] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:50.659011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e474-26a2-421c-8165-7e973b0c3b1b None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:50.659535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e474-26a2-421c-8165-7e973b0c3b1b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:50.659652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e474-26a2-421c-8165-7e973b0c3b1b None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:50.661400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 467/961] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:50.666955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:50.667248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:50.667527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:50.685169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:50.686355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:50.766693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 495/962] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:50.795449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e474-26a2-421c-8165-7e973b0c3b1b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:50.945622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e474-26a2-421c-8165-7e973b0c3b1b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:50.964765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:50.982681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:50.983365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:50.983653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:50.983936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:50.984683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:50.986162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:51.045822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.sql.core [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 04 09:05:51.045822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 04 09:05:51.045822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [parameters: {'user_id': '11eb0494d46a4c9f81163c0571889530', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 04 09:05:51.045822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6cf9ae31-698f-4717-8639-fa6b5f916097 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result = self._query(query) Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application conn.query(q) Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result.read() Jun 04 09:05:51.053220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 04 09:05:51.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application next(self.gen) Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application yield resource Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application next(self.gen) Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.session.rollback() Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.force_reraise() Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise self.value Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application session.commit() Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 04 09:05:51.055134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._prepare_impl() Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.session.flush() Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._flush(objects) Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application compat.raise_( Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application flush_context.execute() Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rec.execute(self) Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 04 09:05:51.056056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 04 09:05:51.058048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result = self._query(query) Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application conn.query(q) Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application result.read() Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 04 09:05:51.059819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [parameters: {'user_id': '11eb0494d46a4c9f81163c0571889530', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:51.060904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 04 09:05:51.061912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:51.063001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 468/963] 10.222.0.46 () {60 vars in 1131 bytes} [Sat Jun 4 09:05:50 2022] POST /identity/v3/users => generated 160 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:51.067577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:51.110629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:51.169468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583e644-c158-440f-8574-fb145c0fde1b None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:51.205489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:51.206203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.206480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.206746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.214719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:51.216186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:51.229694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f449-5259-481b-b549-6f82cc72a330 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.229970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f449-5259-481b-b549-6f82cc72a330 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.230244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f449-5259-481b-b549-6f82cc72a330 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:51.233622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:51.234749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 04 09:05:51.235908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:51.236590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 469/964] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:51.237439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 496/965] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:05:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:51.241778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.242122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.242468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.245610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:51.262004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:51.263984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:51.356893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43445631-a9c5-415a-9330-ffd3b085c6b1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:51.358908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:51.360435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 470/966] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:51.382092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:51.409544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:51.410876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.411626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.412118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.413589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:51.414354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:51.416331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5626c8-f185-4073-8092-e145de17e0b7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:51.455998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 497/967] 10.222.0.46 () {56 vars in 1101 bytes} [Sat Jun 4 09:05:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:51.505990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c6ce-5fcf-4d58-a420-9f316a59b2a9 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:51.532715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:51.545996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355b871-f5ca-4b80-9043-cbfa393cbe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:51.548457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355b871-f5ca-4b80-9043-cbfa393cbe0e None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.548457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355b871-f5ca-4b80-9043-cbfa393cbe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.548457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355b871-f5ca-4b80-9043-cbfa393cbe0e None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.548457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355b871-f5ca-4b80-9043-cbfa393cbe0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:51.549517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355b871-f5ca-4b80-9043-cbfa393cbe0e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:51.640702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 471/968] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:51 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:51.648490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a87c8-8a54-4a33-b85e-19ff893d01e4 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.648842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a87c8-8a54-4a33-b85e-19ff893d01e4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.649107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a87c8-8a54-4a33-b85e-19ff893d01e4 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.651043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 498/969] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:51.657547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.657827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.658151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.678298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:51.679553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:51.776906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 472/970] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:51.815405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a87c8-8a54-4a33-b85e-19ff893d01e4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:51.930369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a87c8-8a54-4a33-b85e-19ff893d01e4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:51.965181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:51.965998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:51.966284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:51.966561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:51.976838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:05:51.985172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:51.986624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:51.987498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 499/971] 10.222.0.46 () {56 vars in 1120 bytes} [Sat Jun 4 09:05:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:51.994160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.097560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:52.103861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470eb96-c19a-4939-a44d-9ed19e4ba01d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:52.141293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:52.141973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.142279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.145437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.145437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:52.145437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:52.145437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:52.163215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 473/972] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:05:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:52.164337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.189732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:52.287273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc9f4e-337a-42c9-ad56-c6ad3bf80bb4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:52.335187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:52.335983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.336258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.336552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.352983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:52.354366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:52.377380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:52.378340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:52.379248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:52.379248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:52.379248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:52.379248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:52.380610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 500/973] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:52.384213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.511287 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac3866-d9d0-443c-a79c-77d26b2d7d6b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:52.545876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:52.546683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.546965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.547241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.548005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:52.548330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:52.549731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:52.564480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 474/974] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:52.567672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.639735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:52.670819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c499ed6-bfc9-40e2-b281-1b79e3c39e0a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:52.705821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:52.707010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.707604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.708076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.716024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:52.717802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-cc103cf1-0d6a-4d4f-b1a9-1c5ecca3f964 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:05:52.733240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:52.734257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 04 09:05:52.735260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:52.741022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 501/975] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:05:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:52.742089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.761981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:52.807648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa20e1-2a82-4755-bc06-d0bd030afcfe None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.808038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa20e1-2a82-4755-bc06-d0bd030afcfe None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.808414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa20e1-2a82-4755-bc06-d0bd030afcfe None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.809769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 502/976] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:52.815468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670fe17-b863-4525-9202-830b823761cb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.815753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670fe17-b863-4525-9202-830b823761cb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.816032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670fe17-b863-4525-9202-830b823761cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.833411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c670fe17-b863-4525-9202-830b823761cb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:52.834596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c670fe17-b863-4525-9202-830b823761cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:52.855693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9d28-26f2-4681-bc35-f381571d6bdd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:52.894378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:52.895050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:52.895340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:52.895596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:52.896465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:52.896847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:52.898280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:52.912328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 503/977] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:52.916950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 475/978] 10.222.0.46 () {56 vars in 1149 bytes} [Sat Jun 4 09:05:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:52.922618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670fe17-b863-4525-9202-830b823761cb None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.935571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:52.986071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.022090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.034473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670fe17-b863-4525-9202-830b823761cb None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.048897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da560d-b82c-4ad2-b443-b090c3e22e3b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.071013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.071672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.071960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.072277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.086701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.087503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.087779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.088062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.088891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.090426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:53.103748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:53.105215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:53.106096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 476/979] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:53.111307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.149222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.150606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.216857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.221488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1847625-6e86-49d3-992e-a0f0d62cb831 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.262976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 504/980] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:05:52 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:05:53.264017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.272613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.273290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.273573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.273849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.274559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:53.274865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.276226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.281753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 477/981] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:53.287341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.292817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.375067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec7a6-9eed-40b5-865f-8f227ff26f1b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.396690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c38bd-f88f-43eb-a7ce-9812c060ef1a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.420506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.421233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.421519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.421800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.433006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.434452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.435043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.435908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.436658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.437419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.438306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:53.438768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.440887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.447805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 478/982] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:53.453271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:05:53.453591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:53.454469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:53.456105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:05:53.456105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:05:53.456105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:05:53.456105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:53.456105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 505/983] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:53.460246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.475890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.527123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.562885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc4c7c-9eff-4d5d-acc1-cce0170efbb3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.575946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f2d3c-5f7c-42ee-823f-5333c1b394dc None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.598635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.599302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.599588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.599871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.600724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] RBAC: Adding query filter params (interface=admin, service_id=728b1aa4b9ea4c9885ccd4c88424119a, region_id=RegionOne) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:53.601044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.602416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.607620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 479/984] 10.222.0.46 () {56 vars in 1249 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/endpoints?service_id=728b1aa4b9ea4c9885ccd4c88424119a&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:53.613859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.616015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.616833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.617327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.617611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.618333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:53.618641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.620011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.632485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 506/985] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:53.637882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.719324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.736833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06830672-b594-46b0-8ed8-2fea9d92812d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.747904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94ba13-23dc-4b83-917e-f645bea547f1 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.767969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:53.774586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.775411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.775702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.775986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.776713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.778091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.792322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 480/986] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:05:53.796756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.797579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.800912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.800912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.807218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.808821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-92c6da10-67fc-4e66-95c4-7ac0401e9224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:05:53.825029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:53.830588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:53.831492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 507/987] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:53.833633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:53.948165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe83791-5a34-43cf-ae22-fa3cdd86738e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:53.984620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:53.985060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:53.985889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:53.985889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:53.986948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:53.987259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:53.989182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.005697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798faf6-f1b3-4363-a489-ac72d257be26 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.005697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798faf6-f1b3-4363-a489-ac72d257be26 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.005697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798faf6-f1b3-4363-a489-ac72d257be26 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.006075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 481/988] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:54.007132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 508/989] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:54.011645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:54.011985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491474f0-7989-4405-bcd9-c14894724596 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.012277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491474f0-7989-4405-bcd9-c14894724596 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.012824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491474f0-7989-4405-bcd9-c14894724596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.031228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-491474f0-7989-4405-bcd9-c14894724596 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:54.032517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-491474f0-7989-4405-bcd9-c14894724596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:54.073539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:54.115722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 509/990] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:54.128101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70572f-69d2-4238-984f-a2b93e48b93d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:54.139899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491474f0-7989-4405-bcd9-c14894724596 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:54.166816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:54.167479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.167866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.168153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.169025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] RBAC: Adding query filter params (role.id=d43e64e54183445db95c7e057d9b7721, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=d1bb3c381d7c47aab9d79db14debc319) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:54.169342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:54.170867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.184312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 482/991] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:05:54 2022] GET /identity/v3/role_assignments?user.id=d1bb3c381d7c47aab9d79db14debc319&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=d43e64e54183445db95c7e057d9b7721 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:54.212245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:54.255643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491474f0-7989-4405-bcd9-c14894724596 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:54.303472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:54.304133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.304453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.304752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.305407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:54.306782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:54.320863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:54.322036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:54.322894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 510/992] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:54.328562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:54.440906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc152-862d-4886-af5b-31bbed04b537 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:54.461200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:54.475999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:54.476701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.476987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.477264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.477970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:54.478275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:54.479641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.489555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 483/993] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:54.491143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:54.591822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715ab0c-6723-4811-ad77-f9e7abcb4b8e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:54.619040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:54.627707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:54.628410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.628686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.635152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.635152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:54.635152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:54.635152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.641363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 511/994] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:54.643568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:54.747326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cb0c-76fe-480a-9a92-0b172c737f56 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:54.782472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:54.783793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.784006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.784372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.785170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] RBAC: Adding query filter params (interface=public, service_id=785299d7c4074a85b6ad1625da358339, region_id=RegionOne) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:54.785485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:54.786867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c816d-3f9c-4a08-91a8-4d94bdba34a3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.796852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 484/995] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:05:54 2022] GET /identity/v3/endpoints?service_id=785299d7c4074a85b6ad1625da358339&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:54.797824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:54.901235 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d9c8-abce-47af-994c-09a70eea7bc8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:54.937628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f797c9-2069-4440-aae5-f288c85926da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:54.938297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f797c9-2069-4440-aae5-f288c85926da None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:54.938580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f797c9-2069-4440-aae5-f288c85926da None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:54.938858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f797c9-2069-4440-aae5-f288c85926da None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:54.939542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f797c9-2069-4440-aae5-f288c85926da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:54.940939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f797c9-2069-4440-aae5-f288c85926da None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:54.952299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 512/996] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:05:54 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:05:56.075727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0d36-a65e-4b77-b07b-848006d9a0c4 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.076533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0d36-a65e-4b77-b07b-848006d9a0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.076533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0d36-a65e-4b77-b07b-848006d9a0c4 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.077933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 485/997] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:56.083580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.084070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.084172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.102801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:56.104327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:56.163053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:56.180327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 513/998] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:56.203223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0d36-a65e-4b77-b07b-848006d9a0c4 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:56.256190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:56.317078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0d36-a65e-4b77-b07b-848006d9a0c4 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:56.358822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:56.359292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.359580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.359850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.360531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:56.361932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-905bb65d-39c1-40ac-86d8-a6092e04a4c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:56.374605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:56.375589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:56.376497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 486/999] 10.222.0.46 () {56 vars in 1126 bytes} [Sat Jun 4 09:05:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:56.383736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:56.465476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067c54a-b72c-4fb9-800f-22b976775b55 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.465827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067c54a-b72c-4fb9-800f-22b976775b55 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.466040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067c54a-b72c-4fb9-800f-22b976775b55 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.467673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 487/1000] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:56.473386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.473674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.473956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.487380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa806d-5cf4-4e2f-bbc3-5e2dfa35cb28 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:56.491546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:56.492778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:56.523631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:56.524307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.524590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.524867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.525561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:56.525863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:56.527229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:56.532333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 514/1001] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:56.538283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:56.566927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 488/1002] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:56.589536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:56.593737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:56.635754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:56.658861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc656b-0898-40ee-806d-f66029d60116 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:56.697548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10a078-c114-4ba4-8b3a-df5863d3ca2c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:56.699086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:56.699751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.700037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.700409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.701123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:56.701431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:56.702802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:56.708902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 515/1003] 10.222.0.46 () {56 vars in 1122 bytes} [Sat Jun 4 09:05:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:56.714460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:56.736608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:56.737411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.737700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.737980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] PATH_INFO: `/v3/services/image` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.738636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:56.740026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:56.753496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:56.754578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:05:56.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 489/1004] 10.222.0.46 () {56 vars in 1105 bytes} [Sat Jun 4 09:05:56 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:05:56.760580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:56.834048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aec01-1d03-4620-9404-f59587d7a716 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:56.851798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:56.872961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ca2a-3a27-408b-8ec4-762c46af3d98 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:56.873646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac8feb-98c4-42ed-9558-e5a803a1dc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:56.874312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac8feb-98c4-42ed-9558-e5a803a1dc0c None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.874591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac8feb-98c4-42ed-9558-e5a803a1dc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.874915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac8feb-98c4-42ed-9558-e5a803a1dc0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.875750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac8feb-98c4-42ed-9558-e5a803a1dc0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:56.877634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac8feb-98c4-42ed-9558-e5a803a1dc0c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:56.883576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:56.913533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:56.914213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:56.914491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:56.914768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:56.915477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] RBAC: Adding query filter params (name=image) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:56.915780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:56.917178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:56.922802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 490/1005] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:56.928206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:57.008749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 516/1006] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:05:57.034609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61afbde-1a27-47b7-9650-45faf9e01968 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:57.081895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:57.082587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.082864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.083122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.083845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] RBAC: Adding query filter params (type=image) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:57.084156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:57.085598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b14e1b-b5b7-44a4-b037-7440d30d4657 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:57.096635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 491/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:57.230280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c577582-437b-486b-8ee0-c8c2c72d400f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.230793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c577582-437b-486b-8ee0-c8c2c72d400f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.230793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c577582-437b-486b-8ee0-c8c2c72d400f None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.232152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 517/1008] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:57.238853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.239410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.239410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.260270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:57.261560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:57.296151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:57.342319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 492/1009] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:57.366504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c577582-437b-486b-8ee0-c8c2c72d400f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:57.476055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c577582-437b-486b-8ee0-c8c2c72d400f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:57.512577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f903-0487-40e5-822a-9606f531052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:57.513245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f903-0487-40e5-822a-9606f531052a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.513524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f903-0487-40e5-822a-9606f531052a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.513798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f903-0487-40e5-822a-9606f531052a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.514446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506f903-0487-40e5-822a-9606f531052a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:57.515814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506f903-0487-40e5-822a-9606f531052a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2506f903-0487-40e5-822a-9606f531052a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:05:57.527896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:05:57.529174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 04 09:05:57.530432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:05:57.531958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 518/1010] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:05:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:05:57.537263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:57.646686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9feebf-f7ee-4b40-8f5c-497b764a8dc7 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:57.681704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:57.682371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.682664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.682940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.683649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:57.683965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:57.685401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:57.690610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 493/1011] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:57.696032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f903-0487-40e5-822a-9606f531052a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:57.739591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:57.803662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f903-0487-40e5-822a-9606f531052a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:57.837953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:57.838616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.838895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.839166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.839866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] RBAC: Adding query filter params (type=compute) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:05:57.840172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:57.841571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabba290-351b-4966-a360-1dbb37a188df None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:57.852617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 519/1012] 10.222.0.46 () {56 vars in 1112 bytes} [Sat Jun 4 09:05:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:05:57.854211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:57.882598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:57.959535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e514e5a-b270-4830-8c71-ea984673a3e2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:57.996568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc935e75-e90b-455d-90b6-ebf12b4c9383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:57.997505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc935e75-e90b-455d-90b6-ebf12b4c9383 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:57.997792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc935e75-e90b-455d-90b6-ebf12b4c9383 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:57.998072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc935e75-e90b-455d-90b6-ebf12b4c9383 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:57.998818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc935e75-e90b-455d-90b6-ebf12b4c9383 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:58.000207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc935e75-e90b-455d-90b6-ebf12b4c9383 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:58.103812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 494/1013] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:05:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:05:59.328638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf38d06-b813-42e6-a72f-132a40534bec None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.329245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf38d06-b813-42e6-a72f-132a40534bec None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.329245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf38d06-b813-42e6-a72f-132a40534bec None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.331274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 520/1014] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:05:59.336535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.336946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.337137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.356377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:59.357652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:59.390162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bf613-dd63-4f88-beab-f743bdefba2f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.390449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bf613-dd63-4f88-beab-f743bdefba2f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.390718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bf613-dd63-4f88-beab-f743bdefba2f None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.392239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 521/1015] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:05:59.397398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.397683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.397960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.416597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:05:59.417804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:05:59.437032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 495/1016] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:59 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:05:59.460587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:59.468622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:59.493562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 522/1017] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:05:59 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:05:59.517715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:59.550610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:59.575531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f8cb8-49dc-453b-9fa0-34e02b55f441 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:59.616851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:59.617661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.617955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.618224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.619416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:59.620954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:59.628034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 496/1018] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:05:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:05:59.628893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:05:59.630799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507264-f10f-4519-9fa6-83f940eb5ff5 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:59.667378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb78c5-925f-4cce-9f58-0c8a4dcf8c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:59.668433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb78c5-925f-4cce-9f58-0c8a4dcf8c5f None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.668905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb78c5-925f-4cce-9f58-0c8a4dcf8c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.669410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb78c5-925f-4cce-9f58-0c8a4dcf8c5f None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.670336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb78c5-925f-4cce-9f58-0c8a4dcf8c5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:59.671898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb78c5-925f-4cce-9f58-0c8a4dcf8c5f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:59.703808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:05:59.728169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738cf26-aa0f-4099-b43e-2be62db02bfc None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:05:59.762167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615f488-bc7b-4504-a850-418a3cc744e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:05:59.762833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f488-bc7b-4504-a850-418a3cc744e5 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:05:59.763117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f488-bc7b-4504-a850-418a3cc744e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:05:59.763395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f488-bc7b-4504-a850-418a3cc744e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:05:59.764110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f488-bc7b-4504-a850-418a3cc744e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:05:59.765580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f488-bc7b-4504-a850-418a3cc744e5 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:05:59.781372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 523/1019] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:59 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:05:59.892396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 497/1020] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:05:59 2022] POST /identity/v3/projects => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:00.517023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b3145-8cac-41f5-a115-d64ca519d98b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:00.517575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b3145-8cac-41f5-a115-d64ca519d98b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:00.517575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b3145-8cac-41f5-a115-d64ca519d98b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:00.518973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 524/1021] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:00.524347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:00.524760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:00.524928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:00.542481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:00.543679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:00.619663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 498/1022] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:00.644950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b3145-8cac-41f5-a115-d64ca519d98b None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:00.739373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:00.750331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b3145-8cac-41f5-a115-d64ca519d98b None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:00.786538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:00.787274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:00.787561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:00.787839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:00.798033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:00.806990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:00.808189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:00.818306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 525/1023] 10.222.0.46 () {56 vars in 1120 bytes} [Sat Jun 4 09:06:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:00.819053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:00.872903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:00.919517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e809-d3b7-4220-b0ac-88afa3ef1b72 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:00.957625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:00.958228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:00.958645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:00.958928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:00.959682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:00.959989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:00.961398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:00.972476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 499/1024] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:06:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:00.975797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:01.086473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b30f8-8566-444e-8a36-4d9dfa255a7c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:01.122116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:01.123173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:01.123449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:01.123718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:01.134827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:01.136275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:01.153308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:01.154355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:01.155381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:01.155381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:01.155381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:01.155381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:01.155381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 526/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:01.161546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:01.263278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9418cb-1967-4311-a140-f64a42fb1196 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:01.281989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:01.304922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:01.305584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:01.305863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:01.306134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:01.306900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:01.307153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:01.308564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:01.321484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 500/1026] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:01.326862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:01.331298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:01.430246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ae99a-4021-448f-a307-b2322d19a56d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:01.464184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:01.465002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:01.465285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:01.465560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:01.473284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:01.474935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:06:01.489556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:01.490570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:06:01.491544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:01.497040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 527/1027] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:06:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:01.497179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:01.599489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61f1c-c7ea-4cb5-83a7-07e5a0402543 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:01.635702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9692b-9835-49db-9303-67a127876166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:01.636736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9692b-9835-49db-9303-67a127876166 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:01.637210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9692b-9835-49db-9303-67a127876166 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:01.637676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9692b-9835-49db-9303-67a127876166 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:01.638721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9692b-9835-49db-9303-67a127876166 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:01.639226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9692b-9835-49db-9303-67a127876166 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:01.640896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9692b-9835-49db-9303-67a127876166 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:01.659906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 501/1028] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:06:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:01.665279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:01.728134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:01.770146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1995d-3074-4672-bd52-eed2f5c8cbac None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:01.808582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2bcf1-2efe-4083-9fa0-2c138bcdb71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:01.809232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bcf1-2efe-4083-9fa0-2c138bcdb71d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:01.809512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bcf1-2efe-4083-9fa0-2c138bcdb71d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:01.809781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bcf1-2efe-4083-9fa0-2c138bcdb71d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:01.884548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2bcf1-2efe-4083-9fa0-2c138bcdb71d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:01.886046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2bcf1-2efe-4083-9fa0-2c138bcdb71d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:01.949651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:01.992490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 528/1029] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:01.993088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9692b-9835-49db-9303-67a127876166 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.026806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.095766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9692b-9835-49db-9303-67a127876166 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.101181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a315-bbcd-4656-bd57-e2443104fe7c None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.101479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a315-bbcd-4656-bd57-e2443104fe7c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.101759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a315-bbcd-4656-bd57-e2443104fe7c None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.103265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 529/1030] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:06:02.108390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b189ff2-cc97-4f97-8ccc-7163671db3ab None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.108678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b189ff2-cc97-4f97-8ccc-7163671db3ab None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.108959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b189ff2-cc97-4f97-8ccc-7163671db3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.125937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7b189ff2-cc97-4f97-8ccc-7163671db3ab None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:02.127609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7b189ff2-cc97-4f97-8ccc-7163671db3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:02.133257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.134068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.134354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.134632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.146000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.147634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:02.165407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:02.166331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:02.167834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:02.167834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:02.167834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:02.167834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:02.167834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 502/1031] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:02.172168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.212338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 530/1032] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:02.224097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4539c3b-27ed-44a8-99a9-d387336ab80d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.224459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4539c3b-27ed-44a8-99a9-d387336ab80d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.225084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4539c3b-27ed-44a8-99a9-d387336ab80d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.227203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 531/1033] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:06:02.232450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.232731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.233305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.251524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:02.253284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:02.273966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.289925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1df4-9ab9-4607-800c-f8e846942959 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.323724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 532/1034] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:06:02.324549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.330955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.331630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.331907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.332191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.332967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:02.333276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.334647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.347496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 503/1035] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:06:02.349234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.352811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.423169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664f7cc-ecb7-4303-a620-f01955d123bd None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.452073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e9e79-f040-4ca3-b2d3-d0982692cf00 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.457220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.458033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.458354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.458638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.459288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.460687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:02.472248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:02.476669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:02.477555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 533/1036] 10.222.0.46 () {56 vars in 1105 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:02.478396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.491672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.492489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.492773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.493042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.494221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.495658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.497316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 504/1037] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:02.498323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.513792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.562011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.573742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2397b8-2553-47f9-9c62-d9e12f06b7a6 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.598972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21090fc7-5cb7-4c92-b195-6fb08164840f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.607089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.607619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.607904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.608191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.615635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.617112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:06:02.631446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:02.632525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 534/1038] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:02.633446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.636327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.637009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.637322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.637576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.638342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:02.638591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.639992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.645435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 505/1039] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:02.646256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.721815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.738512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430902c-4b63-4a93-88da-a1c1667bdc76 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.744414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed0ea-0c7c-470c-b900-8aa7776272f6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.771980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.772607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.772884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.773160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.773858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.775278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.778047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.778705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.778992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.779273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.780050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:02.780459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.781934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.794412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 506/1040] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:02.795299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.896036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a82ab4-1e65-4cc3-9072-4b64c6aee4d6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:02.929127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:02.929788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:02.930075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:02.930355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:02.931055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] RBAC: Adding query filter params (type=image) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:02.931364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:02.932760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:02.934730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:02.937914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 507/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:02.938699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.943597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 535/1042] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:06:02 2022] POST /identity/v3/users => generated 285 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:02.945220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:02.948662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:03.054158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afff2dd-4bca-4f48-92a1-e29c5f99672a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:03.064749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397acc6f-ae7b-4052-ae21-960ec24827ee None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:03.093248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:03.093931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:03.094212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:03.094492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:03.095350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] RBAC: Adding query filter params (role.id=d43e64e54183445db95c7e057d9b7721, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=5c1d511f4c6341e09d54bf8709efdf8e) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:03.095716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:03.099862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:03.111171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 508/1043] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/role_assignments?user.id=5c1d511f4c6341e09d54bf8709efdf8e&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=d43e64e54183445db95c7e057d9b7721 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:03.111474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:03.111474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:03.111474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:03.111474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:03.111835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] RBAC: Adding query filter params (interface=public, service_id=bd8b26c9d0b74fbb8d7f3fa167cf1ab7, region_id=RegionOne) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:03.111835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:03.111835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0565-5901-4c83-803b-1a8e9342e6aa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:03.120773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 536/1044] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:06:02 2022] GET /identity/v3/endpoints?service_id=bd8b26c9d0b74fbb8d7f3fa167cf1ab7&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:03.123239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:03.200557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:03.248766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1532c4e-00b8-4f7c-94c4-a4a405f588fe None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:03.296952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12b26d-d83c-4b67-adc9-238c9024708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:03.297929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12b26d-d83c-4b67-adc9-238c9024708b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:03.298369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12b26d-d83c-4b67-adc9-238c9024708b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:03.298837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12b26d-d83c-4b67-adc9-238c9024708b None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:03.299681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12b26d-d83c-4b67-adc9-238c9024708b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:03.301576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12b26d-d83c-4b67-adc9-238c9024708b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:03.316016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 509/1045] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:06:03 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:06:05.282409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568bc9f-f4a2-4856-9da4-77ae8fe75e2c None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.282885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568bc9f-f4a2-4856-9da4-77ae8fe75e2c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.282965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568bc9f-f4a2-4856-9da4-77ae8fe75e2c None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.285187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 537/1046] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:05.293290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c84f1e-3f08-440a-9af3-32e7c10be17f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.293741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c84f1e-3f08-440a-9af3-32e7c10be17f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.293837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c84f1e-3f08-440a-9af3-32e7c10be17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.310801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-37c84f1e-3f08-440a-9af3-32e7c10be17f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:05.312057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-37c84f1e-3f08-440a-9af3-32e7c10be17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:05.382035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 510/1047] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:05.387594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad97cb1-16fc-48cd-a9d2-0108d23cd76f None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.387891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad97cb1-16fc-48cd-a9d2-0108d23cd76f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.388174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad97cb1-16fc-48cd-a9d2-0108d23cd76f None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.389571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 538/1048] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:05.395289 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.395579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.395858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.406034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad97cb1-16fc-48cd-a9d2-0108d23cd76f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.413940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:05.415128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:05.472005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:05.481520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:05.489609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 511/1049] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:05.511928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.516671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad97cb1-16fc-48cd-a9d2-0108d23cd76f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:05.550919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64f524-fb83-43ed-805c-8970c8b4e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:05.551635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64f524-fb83-43ed-805c-8970c8b4e861 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.551914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64f524-fb83-43ed-805c-8970c8b4e861 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.552188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64f524-fb83-43ed-805c-8970c8b4e861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.563283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64f524-fb83-43ed-805c-8970c8b4e861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-5e64f524-fb83-43ed-805c-8970c8b4e861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:05.571834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:05.572842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:05.573713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 539/1050] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:06:05.574622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba4608-3eda-4280-b854-71162b2fb019 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.574622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba4608-3eda-4280-b854-71162b2fb019 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.574834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba4608-3eda-4280-b854-71162b2fb019 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.576234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 540/1051] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:05.579003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba4608-3eda-4280-b854-71162b2fb019 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.622039 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b875052-1406-4a73-9ae0-4db6974ad46a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:05.657912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb6252-d419-477d-82d6-0e460ccf920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:05.658771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb6252-d419-477d-82d6-0e460ccf920a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.659053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb6252-d419-477d-82d6-0e460ccf920a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.659329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb6252-d419-477d-82d6-0e460ccf920a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.669663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb6252-d419-477d-82d6-0e460ccf920a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-98fb6252-d419-477d-82d6-0e460ccf920a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:05.677973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:05.679108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:05.679985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 512/1052] 10.222.0.46 () {56 vars in 1120 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:05.680737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.680846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.681125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.682188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba4608-3eda-4280-b854-71162b2fb019 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:05.697622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:05.698836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:05.711201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:05.719021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:05.723876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.723876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.723876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.723876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:05.723876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:05.723876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:05.730935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 541/1053] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:06:05.731740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.743927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:05.767402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 513/1054] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:05.768888 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.831825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc13dc-bb78-44a4-981c-a8743a99768c None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:05.864160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:05.864851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.865136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.865415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.866143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:05.866450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:05.866878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522601b5-b5b8-4d6a-91fd-ff804a5dda1e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:05.867871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:05.876542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 542/1055] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:05.878380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.903966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:05.904889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:05.905172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:05.905450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] PATH_INFO: `/v3/users/0d612fea8c6d44839fa333d7be9431d7` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:05.919077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] RBAC: Authorizing `identity:get_user(user_id=0d612fea8c6d44839fa333d7be9431d7)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:05.920687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:05.927573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 514/1056] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/users/0d612fea8c6d44839fa333d7be9431d7 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:05.927573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:05.934090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:05.959079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:05.981870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc13d-4352-459f-a2b1-c40ff9b2b590 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.015414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.016196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.016508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.016791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] PATH_INFO: `/v3/services/image` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.017433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.019361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.023515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376306f-37c0-4861-9638-e6bc8bd10173 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:06.030992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:06.032148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:06.033248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 543/1057] 10.222.0.46 () {56 vars in 1105 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:06.034063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.060995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.061839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.062119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.062395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.072678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.074143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:06.092595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:06.093564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:06.094477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:06.094477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:06.094477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:06.094477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:06.094477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 515/1058] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:06.094880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.138772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524d30a-1415-4a37-96b4-05ca36b2a823 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.156602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.175736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.176420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.176709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.176992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.184073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.195401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7271d-0cd2-4940-8dcc-3f3bc33aaff6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.320973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.321650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.321910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.322188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.322890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] RBAC: Adding query filter params (name=image) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:06.323203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.324732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.329857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 516/1059] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:06.331373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.358004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.359403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.430800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f47eca-f155-4d04-8fb0-38ef2c85458c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.458587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 544/1060] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:06.459776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.471932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.472660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.472908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.476275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.476275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:06.476275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.476275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.478761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.487195 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 517/1061] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:06.488011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.503754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.560736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0893-f21d-4e0b-a949-c4417bbf1088 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.588032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54774cdd-11ed-455f-bcd6-6a9a5798d13f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.596197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.596896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.597181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.597461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.598176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] RBAC: Adding query filter params (type=image) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:06.598482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.599866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.604954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 545/1062] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:06.605746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.628655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.629445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.629727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.630005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.631226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=d1dbe1b9d42b48169085f5d7ec6ed094)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.633709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.634445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 518/1063] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:06.635196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.693818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.712365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406b3fe-66f8-4726-bc71-b8a0920cac23 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.718086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.737611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae9dd3-5cde-4ef2-9ec2-f069982e1bd6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.746235 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.747014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.747290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.747823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.756210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.757660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.772036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:06:06.772692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:06.774591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:06:06.776514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:06.777540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.777540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.777540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.777889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 546/1064] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:06.778018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.778148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.778292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.867343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 519/1065] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:06:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:06.868185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:06.878428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463da0f-273b-48d3-b3dc-b4ac7a1aba08 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:06.926378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:06.927047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:06.927326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:06.927598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:06.928465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094, user.id=0d612fea8c6d44839fa333d7be9431d7) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:06.928776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:06.930239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:06.933533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:06.936871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 547/1066] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/role_assignments?user.id=0d612fea8c6d44839fa333d7be9431d7&scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:07.007037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab581f4-7f2e-4ebd-95da-1cf0956c29d8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:07.047731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:07.048475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:07.048761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:07.049293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:07.049946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:07.050251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:07.051748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:07.055025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:07.078494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 520/1067] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:06:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:07.080073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:07.191948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05110e95-6fd2-4421-bcc5-3551284da264 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:07.228440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:07.229230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:07.229513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:07.229791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:07.240957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:07.242414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:07.261251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:07.262400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:07.263536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:07.263536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:07.263536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:07.263536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:07.263536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 548/1068] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:06:07.270922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:07.377837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72499a-c429-41ef-ba96-769837d840d2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:07.412398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:07.413071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:07.413353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:07.413628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:07.414354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:07.414649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:07.416029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:07.432880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 521/1069] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:07.434160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:07.492739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:07.540683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55377bc4-33a4-4c14-be47-67b95c0ea5f9 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:07.574721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:07.575508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:07.575784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:07.576058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:07.582628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:07.584054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:06:07.597897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:07.598828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:06:07.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:07.600941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 549/1070] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:06:07.604816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:07.667136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:07.711611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd517fe-b8fa-4e05-8266-9adb5e7c408b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:07.745150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:07.745728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:07.746005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:07.746277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:07.747054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:07.747361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:07.749078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3005-77ee-4868-ae49-be14931b9307 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:07.762084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 522/1071] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:07.767797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:07.867050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bf576-3837-4cce-a329-d6d7d7fd5b79 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:07.880212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:07.900400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed94c3-10c7-4b47-9495-3af44bd7c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:07.901282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed94c3-10c7-4b47-9495-3af44bd7c71b None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:07.901630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed94c3-10c7-4b47-9495-3af44bd7c71b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:07.901821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed94c3-10c7-4b47-9495-3af44bd7c71b None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/5c1d511f4c6341e09d54bf8709efdf8e/roles/d43e64e54183445db95c7e057d9b7721` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:07.903995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed94c3-10c7-4b47-9495-3af44bd7c71b None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=5c1d511f4c6341e09d54bf8709efdf8e, role_id=d43e64e54183445db95c7e057d9b7721)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:07.905449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed94c3-10c7-4b47-9495-3af44bd7c71b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:07.975034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 550/1072] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:06:07 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/5c1d511f4c6341e09d54bf8709efdf8e/roles/d43e64e54183445db95c7e057d9b7721 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:06:09.219109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dddb-81fa-4a15-8f4a-47a5b89a8517 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.219642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dddb-81fa-4a15-8f4a-47a5b89a8517 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.219642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dddb-81fa-4a15-8f4a-47a5b89a8517 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.222122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 523/1073] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:09.226790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.227242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.227341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.243558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadd5e-a9b0-4908-a7e5-2a3f0350516d None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.243902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadd5e-a9b0-4908-a7e5-2a3f0350516d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.244124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadd5e-a9b0-4908-a7e5-2a3f0350516d None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.244769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:09.248333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 524/1074] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:09.248474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:09.251112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.251399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.251679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.268938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:09.270472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:09.331735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 551/1075] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:09.341224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 525/1076] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:06:09.354591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.365733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.413100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:09.462938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d378e-b68b-4165-8de0-efb93ac61767 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:09.464616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:09.479218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691038-15f1-4e22-ac23-5571de887363 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:09.499484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:09.500600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.500883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.501158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.511654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:09.511832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:09.512703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.513198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.513677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:09.519293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:09.520222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:09.521354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 552/1077] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:09.526461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.595468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:09.597514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:09.633035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0db6a-2128-41a8-85c4-5fb89cad58df None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:09.652797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:09.669812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:09.670480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.670765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.671046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.671792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:09.672131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:09.673515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:09.682566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 553/1078] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:06:09.688194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.716782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:09.720118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 526/1079] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 04 09:06:09.721116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.793381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94668c9-4726-49ac-82ff-b94063607e38 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:09.829855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:09.830632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a10d88-d986-4258-b236-f6b61c9b21b3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:09.830832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.831065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.831339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] PATH_INFO: `/v3/users/0d612fea8c6d44839fa333d7be9431d7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.832592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] RBAC: Authorizing `identity:get_user(user_id=0d612fea8c6d44839fa333d7be9431d7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:09.834037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:09.840558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 554/1080] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/users/0d612fea8c6d44839fa333d7be9431d7 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:06:09.841225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.870469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:09.871270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.871556 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.871834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.884821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:09.886527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:09.889699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:09.908377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:09.909497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:09.910409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:09.910409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:09.910409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:09.910409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:09.910409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 527/1081] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:09.915662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:09.945859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:09.956205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceda22-b025-40ee-8ac0-5d59cf26692d None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:09.995440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:09.996330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:09.996621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:09.996900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:09.998758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] RBAC: Authorizing `identity:get_project(project_id=d1dbe1b9d42b48169085f5d7ec6ed094)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:10.000648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:10.003437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 555/1082] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:06:10.008993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:10.023851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef217a6-46b6-4e17-bbcd-29b5eccf5b3f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:10.061784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:10.062700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.062972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.063248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.063974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:10.064275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:10.065710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b705c-5e44-4ca3-a9de-2ae0b8f15ed9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:10.078439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 528/1083] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:10.125821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e722951-e9ac-4c63-a5b6-b19cffd37722 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:10.165480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:10.166391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.166670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.166946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094/users/0d612fea8c6d44839fa333d7be9431d7/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.169055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1dbe1b9d42b48169085f5d7ec6ed094, user_id=0d612fea8c6d44839fa333d7be9431d7, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:10.170507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:10.220478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ec833-e073-49be-884b-73f9ed3722a0 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.221572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ec833-e073-49be-884b-73f9ed3722a0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.221572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ec833-e073-49be-884b-73f9ed3722a0 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.222998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 529/1084] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:06:10.228871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.229162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.229444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.246945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 556/1085] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:06:10 2022] PUT /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094/users/0d612fea8c6d44839fa333d7be9431d7/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:06:10.249572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:10.250838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:10.322879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:10.341129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 530/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:06:10.366065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:10.392311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:10.476896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc552c-3f44-45eb-85d6-856e4c6945b7 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:10.511016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:10.511814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.512091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.512391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.522396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:10.530540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:10.531510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 04 09:06:10.532394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:10.536788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 557/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Sat Jun 4 09:06:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:10.537703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:10.634621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4446b3-78f7-43d3-9708-353e86b4752f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:10.669505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:10.670162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.670439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.670710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.671441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:10.671740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:10.673128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:10.680937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 531/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:06:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:10.686701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:10.770064 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:10.789955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3519e0-c537-4af1-8cc3-f126dd9f042a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:10.823172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:10.823953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.824229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.824536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.835715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:10.837182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:10.853749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:10.854839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:10.860682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:10.860682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:10.860682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:10.860682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:10.860682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 558/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:06:10.861035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:10.909663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:10.959372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f655091-1bd0-417f-8273-d503a6a47629 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:10.992115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:10.992809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:10.993092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:10.993367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:10.994084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:10.994388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:10.995783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:11.008494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 532/1090] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:11.014115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:11.112736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a6c0b-3bde-4719-a9ff-18c1efcc6f23 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:11.140787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:11.146771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:11.147520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:11.147802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:11.148078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:11.155649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:11.157239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:06:11.171531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:11.172683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:11.173871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 559/1091] 10.222.0.46 () {56 vars in 1093 bytes} [Sat Jun 4 09:06:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:06:11.181147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:11.280015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375fc03-35c0-412e-94c5-befa948b1137 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:11.290006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:11.317653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:11.318324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:11.318610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:11.318889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:11.319725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:11.320030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:11.321499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:11.343618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 533/1092] 10.222.0.46 () {56 vars in 1133 bytes} [Sat Jun 4 09:06:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:11.345020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:11.449116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dc233-c2ec-41d1-ae20-d3e72bde7883 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:11.481454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:11.482101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:11.482385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:11.482660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:11.540520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:11.550508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:11.551963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:11.648402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 560/1093] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:11.654136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:11.751524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da3a7-9ec8-419c-a818-4eef8e35a50e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:11.785799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:11.786591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:11.786876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:11.787151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:11.797199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:11.798642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:11.816417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:11.817502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:11.818710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:06:11.818710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:11.818710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:06:11.818710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:11.818710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 534/1094] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:11.823307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:11.913189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:11.921558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac33b83-2c4c-437a-8d8a-fb71fca7dab0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:11.956079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:11.956766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:11.957047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:11.957322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:11.958035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:11.958339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:11.959703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:11.970681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 561/1095] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:11.976220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:11.995334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:12.076055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fd2bf-0252-44fe-80ca-a2b749793386 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:12.108748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:12.109534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.109817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.110092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.117138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:12.118572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:06:12.132835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:12.133758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:12.134655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 535/1096] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:06:12.139841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:12.241965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd14c10-a646-473b-9626-4cc73e02a817 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:12.277462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:12.278124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.278406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.278685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.279465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:12.279774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:12.281231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efafd39-d859-4529-a4be-ce4278f396c7 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:12.296564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 562/1097] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:12.303932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:12.384686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:12.405705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2d5a-33db-46c4-850c-506a1daf194f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:12.438174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:12.438823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.439102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.439379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.440719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] RBAC: Adding query filter params (role.id=d43e64e54183445db95c7e057d9b7721, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=5c1d511f4c6341e09d54bf8709efdf8e) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:12.441025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:12.442445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75e924-c2d4-4a95-98ed-82bf063d0590 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:12.448297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 536/1098] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:12 2022] GET /identity/v3/role_assignments?user.id=5c1d511f4c6341e09d54bf8709efdf8e&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=d43e64e54183445db95c7e057d9b7721 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:12.490562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d32aea-fc38-4f6e-9d9f-351d6b62e652 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.491139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d32aea-fc38-4f6e-9d9f-351d6b62e652 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.491139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d32aea-fc38-4f6e-9d9f-351d6b62e652 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.492505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 563/1099] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:12.498670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.498961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.499236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.517616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:12.519450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:12.591292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 537/1100] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:12.615145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d32aea-fc38-4f6e-9d9f-351d6b62e652 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:12.631837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:12.725981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d32aea-fc38-4f6e-9d9f-351d6b62e652 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:12.761341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:12.762134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.762418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.762693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.773672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:12.781594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:12.782569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:12.783418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 564/1101] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:12.788966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:12.862839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:12.894041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd716d-00e7-4bda-9ad8-df2f5ece79c3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:12.930669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:12.931350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:12.931633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:12.931913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:12.932702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:12.933013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:12.934418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:12.943916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 538/1102] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:12.950604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:13.029872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:13.066014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e817cef-da4a-4241-a3ec-92dd9b057290 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:13.102120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:13.103282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:13.103561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:13.103836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] PATH_INFO: `/v3/users/0d612fea8c6d44839fa333d7be9431d7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:13.105266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] RBAC: Authorizing `identity:get_user(user_id=0d612fea8c6d44839fa333d7be9431d7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:13.107895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:13.108537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 565/1103] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:12 2022] GET /identity/v3/users/0d612fea8c6d44839fa333d7be9431d7 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:13.115340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:13.223224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1a6c4-c544-4204-9eff-12dcfea96825 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:13.258548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:13.259224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:13.259494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:13.259783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:13.262495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:13.332551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:13.334046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:13.435481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 539/1104] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:13.441362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:13.539938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dcceb-d242-404f-848f-74d5e61b90b2 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:13.572418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b037e9-f5c1-4360-aa57-2abbfb4504fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:13.573265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b037e9-f5c1-4360-aa57-2abbfb4504fd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:13.573544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b037e9-f5c1-4360-aa57-2abbfb4504fd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:13.573816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b037e9-f5c1-4360-aa57-2abbfb4504fd None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:13.574994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b037e9-f5c1-4360-aa57-2abbfb4504fd None admin] RBAC: Authorizing `identity:get_project(project_id=d1dbe1b9d42b48169085f5d7ec6ed094)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:13.576450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b037e9-f5c1-4360-aa57-2abbfb4504fd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:13.578138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 566/1105] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:13 2022] GET /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:13.585114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:13.617896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:13.682536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204466-2f87-4e36-8717-7459b534816e None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:13.715654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:13.716323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:13.716615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:13.716894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:13.717726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094, user.id=0d612fea8c6d44839fa333d7be9431d7) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:13.718043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:13.719465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d091-d153-46e7-911f-2f491b6803a2 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:13.725919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 540/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:13 2022] GET /identity/v3/role_assignments?user.id=0d612fea8c6d44839fa333d7be9431d7&scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:15.980964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b171a1-0eb5-4920-bf39-634c59418ac8 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:15.981501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b171a1-0eb5-4920-bf39-634c59418ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:15.981501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b171a1-0eb5-4920-bf39-634c59418ac8 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:15.982910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 567/1107] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:15.988656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:15.989039 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:15.989206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:16.006311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:16.007491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:16.071330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 541/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:16.095631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b171a1-0eb5-4920-bf39-634c59418ac8 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:16.107283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:16.195751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b171a1-0eb5-4920-bf39-634c59418ac8 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:16.229119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4c7ba-5aa2-42df-afcb-00adbc69349b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:16.229898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4c7ba-5aa2-42df-afcb-00adbc69349b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:16.230180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4c7ba-5aa2-42df-afcb-00adbc69349b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:16.230455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4c7ba-5aa2-42df-afcb-00adbc69349b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:16.231657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4c7ba-5aa2-42df-afcb-00adbc69349b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:16.233054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4c7ba-5aa2-42df-afcb-00adbc69349b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:16.234660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 568/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:16.241076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:16.268585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:16.338960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ae10-cda3-416c-829c-abceb4bfcd2d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:16.374344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954d45e-a4c3-40a4-8d80-e4eff63f8229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:16.375014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954d45e-a4c3-40a4-8d80-e4eff63f8229 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:16.375300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954d45e-a4c3-40a4-8d80-e4eff63f8229 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:16.375579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954d45e-a4c3-40a4-8d80-e4eff63f8229 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:16.376322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954d45e-a4c3-40a4-8d80-e4eff63f8229 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:16.377824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954d45e-a4c3-40a4-8d80-e4eff63f8229 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:16.483701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 542/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:06:16 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:18.779997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c3af1-d91b-4fe4-bdc3-d61bf8a90024 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:18.780555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c3af1-d91b-4fe4-bdc3-d61bf8a90024 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:18.780555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c3af1-d91b-4fe4-bdc3-d61bf8a90024 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:18.781939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 569/1111] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:18.787252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:18.787696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:18.787814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:18.805056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:18.806267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:18.871870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 543/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:18.896211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c3af1-d91b-4fe4-bdc3-d61bf8a90024 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:18.970837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:18.993947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c3af1-d91b-4fe4-bdc3-d61bf8a90024 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:19.027163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:19.028009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:19.028320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:19.028599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] PATH_INFO: `/v3/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:19.030131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] RBAC: Authorizing `identity:get_role(role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:19.031452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:19.032858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 570/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:18 2022] GET /identity/v3/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:19.039187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:19.054531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:19.135673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f97ed-6345-44b4-8af9-d18ac6f3b27b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:19.168244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:19.169128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:19.169406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:19.169677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] PATH_INFO: `/v3/users/8e4954049bf84fc5ac8d2e40e3ac5784` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:19.183009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] RBAC: Authorizing `identity:get_user(user_id=8e4954049bf84fc5ac8d2e40e3ac5784)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:19.184483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:19.186121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 544/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:19 2022] GET /identity/v3/users/8e4954049bf84fc5ac8d2e40e3ac5784 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:19.191918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:19.287879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa5bda-5289-4e9c-a576-0e14573573ed None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:19.320174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:19.320837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:19.321118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:19.321390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:19.335037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:19.385892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:19.387310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:19.483916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 571/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:19.489854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:19.557025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:19.584739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d25f-943a-4263-9ec7-f5ae24de7be6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:19.617654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ce2a3-d164-42cc-8c94-81ab28910084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:19.618496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ce2a3-d164-42cc-8c94-81ab28910084 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:19.618779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ce2a3-d164-42cc-8c94-81ab28910084 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:19.619054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ce2a3-d164-42cc-8c94-81ab28910084 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:19.620223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ce2a3-d164-42cc-8c94-81ab28910084 None admin] RBAC: Authorizing `identity:get_project(project_id=d1dbe1b9d42b48169085f5d7ec6ed094)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:19.621674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ce2a3-d164-42cc-8c94-81ab28910084 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:19.623635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 545/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:19 2022] GET /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:19.629857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:19.674445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:19.726210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489a492-dedc-4321-a76a-92bd5ec9a8ae None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:19.758619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:19.759257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:19.759541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:19.759816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:19.760666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] RBAC: Adding query filter params (role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094, user.id=8e4954049bf84fc5ac8d2e40e3ac5784) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:19.760978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:19.762410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78956060-81c2-4ac8-9327-e39569cc3cf4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:19.767757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 572/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:19 2022] GET /identity/v3/role_assignments?user.id=8e4954049bf84fc5ac8d2e40e3ac5784&scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:21.983534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104d642-9c4a-4519-8cea-d4db88fbe883 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:21.984139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104d642-9c4a-4519-8cea-d4db88fbe883 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:21.984139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104d642-9c4a-4519-8cea-d4db88fbe883 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:21.985467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 546/1118] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:21.991131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:21.991560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:21.991672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:22.008840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:22.010032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:22.073547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 573/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:22.095519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104d642-9c4a-4519-8cea-d4db88fbe883 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:22.193785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104d642-9c4a-4519-8cea-d4db88fbe883 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:22.226966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:22.227820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:22.228099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:22.228404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] PATH_INFO: `/v3/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:22.229879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] RBAC: Authorizing `identity:get_role(role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:22.231252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:22.232658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 547/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:22 2022] GET /identity/v3/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:22.239182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:22.281587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:22.340139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d129643-c469-4552-9cb7-3d7f540054ae None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:22.373244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:22.374105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:22.374388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:22.374665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] PATH_INFO: `/v3/users/8e4954049bf84fc5ac8d2e40e3ac5784` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:22.375839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=8e4954049bf84fc5ac8d2e40e3ac5784)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:22.377292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:22.380256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 574/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:22 2022] GET /identity/v3/users/8e4954049bf84fc5ac8d2e40e3ac5784 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:22.386005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:22.397711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:22.486108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38743a1-7eaa-438e-9f4c-cb70eb62c48c None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:22.520371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f394679-d19d-469a-b261-e96266a81f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:22.521174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f394679-d19d-469a-b261-e96266a81f74 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:22.521459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f394679-d19d-469a-b261-e96266a81f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:22.521737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f394679-d19d-469a-b261-e96266a81f74 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:22.522918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f394679-d19d-469a-b261-e96266a81f74 None admin] RBAC: Authorizing `identity:get_project(project_id=d1dbe1b9d42b48169085f5d7ec6ed094)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:22.524393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f394679-d19d-469a-b261-e96266a81f74 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:22.526094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 548/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:22 2022] GET /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:22.532169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:22.626963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275af83d-13a8-42f1-89cd-810b790a9bd0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:22.634173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:22.661252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af7e7d-4a1d-4822-ad45-29793580c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:22.662122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af7e7d-4a1d-4822-ad45-29793580c577 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:22.662400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af7e7d-4a1d-4822-ad45-29793580c577 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:22.662671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af7e7d-4a1d-4822-ad45-29793580c577 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094/users/8e4954049bf84fc5ac8d2e40e3ac5784/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:22.664846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af7e7d-4a1d-4822-ad45-29793580c577 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1dbe1b9d42b48169085f5d7ec6ed094, user_id=8e4954049bf84fc5ac8d2e40e3ac5784, role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:22.666260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af7e7d-4a1d-4822-ad45-29793580c577 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:22.740268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 575/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:06:22 2022] PUT /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094/users/8e4954049bf84fc5ac8d2e40e3ac5784/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:06:24.949481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db4b23-51bf-45ca-b56b-782e64bcf7f9 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:24.950015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db4b23-51bf-45ca-b56b-782e64bcf7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:24.950015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db4b23-51bf-45ca-b56b-782e64bcf7f9 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:24.951373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 549/1124] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:24.956552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:24.956953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:24.957110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:24.972977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:24.974161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:25.039373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 576/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:25.062874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db4b23-51bf-45ca-b56b-782e64bcf7f9 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:25.139659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:25.163102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db4b23-51bf-45ca-b56b-782e64bcf7f9 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:25.197074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:25.197926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:25.198213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:25.198493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] PATH_INFO: `/v3/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:25.200041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] RBAC: Authorizing `identity:get_role(role_id=49b0cdd8c5bf44f19239b3d3e3ad83a8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:25.201403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:25.202790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 550/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:25 2022] GET /identity/v3/roles/49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:25.208554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:25.299733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:25.305965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be9082-c3df-4957-b33a-b7c279533105 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:25.338799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:25.339662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:25.339941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:25.340211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] PATH_INFO: `/v3/users/8e4954049bf84fc5ac8d2e40e3ac5784` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:25.341410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] RBAC: Authorizing `identity:get_user(user_id=8e4954049bf84fc5ac8d2e40e3ac5784)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:25.342831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:25.344457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 577/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:25 2022] GET /identity/v3/users/8e4954049bf84fc5ac8d2e40e3ac5784 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:25.350804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:25.447338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7403c6-c024-4746-90c1-973e6072cd49 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:25.480779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:25.481350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:25.481634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:25.481912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:25.495728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:25.547835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:25.549316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:25.642938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 551/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:25.649354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:25.744381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2257-e5ff-496d-b0df-0eaabe53a1b5 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:25.776757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7629c47-9b36-4511-b450-c6ce18c52630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:25.777599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629c47-9b36-4511-b450-c6ce18c52630 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:25.777876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629c47-9b36-4511-b450-c6ce18c52630 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:25.778149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629c47-9b36-4511-b450-c6ce18c52630 None admin] PATH_INFO: `/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:25.779298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7629c47-9b36-4511-b450-c6ce18c52630 None admin] RBAC: Authorizing `identity:get_project(project_id=d1dbe1b9d42b48169085f5d7ec6ed094)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:25.780755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7629c47-9b36-4511-b450-c6ce18c52630 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:25.782418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 578/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:25 2022] GET /identity/v3/projects/d1dbe1b9d42b48169085f5d7ec6ed094 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:25.789413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:25.834208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:25.887125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3995d-47a3-413e-bc2b-9e2721354f51 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:25.920324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:25.920981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:25.921266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:25.921548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:25.922376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] RBAC: Adding query filter params (role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8, scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094, user.id=8e4954049bf84fc5ac8d2e40e3ac5784) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:25.922690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:25.924118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c5421-c8df-44a8-97c9-b415fed68c7d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:25.929931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 552/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:25 2022] GET /identity/v3/role_assignments?user.id=8e4954049bf84fc5ac8d2e40e3ac5784&scope.project.id=d1dbe1b9d42b48169085f5d7ec6ed094&role.id=49b0cdd8c5bf44f19239b3d3e3ad83a8 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:28.166007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dcb42-a314-46f7-8557-79eb45bb290c None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:28.166547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dcb42-a314-46f7-8557-79eb45bb290c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:28.166637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dcb42-a314-46f7-8557-79eb45bb290c None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:28.168023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 579/1131] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:28.173455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:28.173925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:28.174029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:28.193330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:28.194537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:28.260052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 553/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:28.283995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dcb42-a314-46f7-8557-79eb45bb290c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:28.316741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:28.386182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dcb42-a314-46f7-8557-79eb45bb290c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:28.419962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a8176-fae7-4144-9d13-35d48be91412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:28.420702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a8176-fae7-4144-9d13-35d48be91412 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:28.420988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a8176-fae7-4144-9d13-35d48be91412 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:28.421263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a8176-fae7-4144-9d13-35d48be91412 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:28.422437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a8176-fae7-4144-9d13-35d48be91412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:28.423867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a8176-fae7-4144-9d13-35d48be91412 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:28.425545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 580/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:28.431310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:28.470570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:28.541166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf4fa9-8825-434d-8393-dcdc1ca76ded None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:28.575858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd083ee7-4e22-401b-a410-1d9a40253085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:28.576538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd083ee7-4e22-401b-a410-1d9a40253085 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:28.576824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd083ee7-4e22-401b-a410-1d9a40253085 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:28.577101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd083ee7-4e22-401b-a410-1d9a40253085 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:28.577823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd083ee7-4e22-401b-a410-1d9a40253085 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:28.579276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd083ee7-4e22-401b-a410-1d9a40253085 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:28.668543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 554/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:06:28 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:30.918978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f17a-8067-4200-ac5f-08dd26432d85 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:30.919537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f17a-8067-4200-ac5f-08dd26432d85 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:30.919663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f17a-8067-4200-ac5f-08dd26432d85 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:30.921067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 581/1135] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:30.926863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:30.927272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:30.927419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:30.944299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:30.945488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:31.009935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 555/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 04 09:06:31.035334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038f17a-8067-4200-ac5f-08dd26432d85 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:31.129220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:31.132959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038f17a-8067-4200-ac5f-08dd26432d85 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:31.166020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5182737-9eb6-42c6-888f-ec4965d81225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:31.166787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182737-9eb6-42c6-888f-ec4965d81225 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:31.167063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182737-9eb6-42c6-888f-ec4965d81225 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:31.167330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182737-9eb6-42c6-888f-ec4965d81225 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:31.168495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5182737-9eb6-42c6-888f-ec4965d81225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:31.169968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5182737-9eb6-42c6-888f-ec4965d81225 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:31.171515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 582/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:06:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:31.177839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:31.198252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:31.274680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb85b-3642-4599-99ae-d0189771a30e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:31.308885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853a06a-d12a-4434-a88a-7db0deb44ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:31.309558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853a06a-d12a-4434-a88a-7db0deb44ff9 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:31.309843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853a06a-d12a-4434-a88a-7db0deb44ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:31.310120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853a06a-d12a-4434-a88a-7db0deb44ff9 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:31.310866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853a06a-d12a-4434-a88a-7db0deb44ff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:31.312314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853a06a-d12a-4434-a88a-7db0deb44ff9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:31.415875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 556/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:06:31 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:33.671651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e231-7ef6-4720-9c7b-efbec510bcf4 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:33.672247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e231-7ef6-4720-9c7b-efbec510bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:33.672247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e231-7ef6-4720-9c7b-efbec510bcf4 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:33.673553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 583/1139] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:33.679269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11679122-3168-4507-9978-47d32afa9378 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:33.679714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11679122-3168-4507-9978-47d32afa9378 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:33.679823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11679122-3168-4507-9978-47d32afa9378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:33.696765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-11679122-3168-4507-9978-47d32afa9378 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:33.697953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-11679122-3168-4507-9978-47d32afa9378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:33.762728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 557/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:33.785869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e231-7ef6-4720-9c7b-efbec510bcf4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:33.884811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e231-7ef6-4720-9c7b-efbec510bcf4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:33.916913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:33.917689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:33.918030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:33.918309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:33.928637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:33.936098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:33.937325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:33.938477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 584/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:33.944082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11679122-3168-4507-9978-47d32afa9378 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:33.952046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:34.038744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11679122-3168-4507-9978-47d32afa9378 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:34.071192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:34.071845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:34.072123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:34.072424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:34.073158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:34.073463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:34.074823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:34.082614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 558/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:34.088522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:34.117249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:34.187099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347b5ef-1f69-4f30-b29a-1776941a42c6 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:34.219176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:34.220035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:34.220344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:34.220627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] PATH_INFO: `/v3/users/a0039171b32648cf9664a9f4692e22f6` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:34.233926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] RBAC: Authorizing `identity:get_user(user_id=a0039171b32648cf9664a9f4692e22f6)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:34.235370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:34.237043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 585/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:34 2022] GET /identity/v3/users/a0039171b32648cf9664a9f4692e22f6 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:34.243531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:34.308823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:34.340835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fc105-61fd-4916-ae3f-2a67bc0a5010 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:34.374746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:34.375394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:34.375675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:34.375950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:34.440970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:34.442365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:34.505878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:34.539107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 559/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:34.545090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:34.635664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:34.650566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b934-a6f2-43b2-835a-7d97f5abee45 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:34.684138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e67ad-0fdd-4258-8b25-70fe0e46e200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:34.685025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e67ad-0fdd-4258-8b25-70fe0e46e200 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:34.685303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e67ad-0fdd-4258-8b25-70fe0e46e200 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:34.685575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e67ad-0fdd-4258-8b25-70fe0e46e200 None admin] PATH_INFO: `/v3/projects/13058ee67b994090a55901ddc37db78e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:34.686733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e67ad-0fdd-4258-8b25-70fe0e46e200 None admin] RBAC: Authorizing `identity:get_project(project_id=13058ee67b994090a55901ddc37db78e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:34.688151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e67ad-0fdd-4258-8b25-70fe0e46e200 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:34.689870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 586/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:34 2022] GET /identity/v3/projects/13058ee67b994090a55901ddc37db78e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:34.696245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:34.793109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42263406-488d-47a3-b77d-567511c2ff26 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:34.826335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:34.826973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:34.827252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:34.827522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:34.828361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=13058ee67b994090a55901ddc37db78e, user.id=a0039171b32648cf9664a9f4692e22f6) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:34.828671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:34.830098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869e9ea-1eca-4172-9bcf-eb483628d03b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:34.836018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 560/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:34 2022] GET /identity/v3/role_assignments?user.id=a0039171b32648cf9664a9f4692e22f6&scope.project.id=13058ee67b994090a55901ddc37db78e&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:37.041777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fc7e5-f050-47e3-a770-894cfdac721e None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.042341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fc7e5-f050-47e3-a770-894cfdac721e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.042341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fc7e5-f050-47e3-a770-894cfdac721e None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.043666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 587/1147] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:37.048837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.049237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.049394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.066267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:37.067464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:37.092070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:37.131931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 561/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:37.154214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fc7e5-f050-47e3-a770-894cfdac721e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:37.253883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fc7e5-f050-47e3-a770-894cfdac721e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:37.287010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:37.287792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.288075 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.288377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.298501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:37.306735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:37.307818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:37.308728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 588/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:37.313913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:37.408812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd99de4-99b7-4575-bb78-612e8bd9812a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:37.441474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:37.442125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.442408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.442683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.443423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:37.443726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:37.445120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:37.452937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 562/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:37.459541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:37.475970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:37.556698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f52a9-208e-4cb0-a608-1e25882fe337 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:37.589151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:37.590001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.590281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.590553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] PATH_INFO: `/v3/users/a0039171b32648cf9664a9f4692e22f6` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.591746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] RBAC: Authorizing `identity:get_user(user_id=a0039171b32648cf9664a9f4692e22f6)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:37.593202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:37.594825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 589/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:37 2022] GET /identity/v3/users/a0039171b32648cf9664a9f4692e22f6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:37.601383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:37.626823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:37.698855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dfc4f-7db0-4c34-b3bc-0ba3ff4b3b96 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:37.732189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006eb20-0837-430f-b3b4-4903cdede754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:37.733070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006eb20-0837-430f-b3b4-4903cdede754 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.733356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006eb20-0837-430f-b3b4-4903cdede754 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.733633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006eb20-0837-430f-b3b4-4903cdede754 None admin] PATH_INFO: `/v3/projects/13058ee67b994090a55901ddc37db78e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.734806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006eb20-0837-430f-b3b4-4903cdede754 None admin] RBAC: Authorizing `identity:get_project(project_id=13058ee67b994090a55901ddc37db78e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:37.736249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006eb20-0837-430f-b3b4-4903cdede754 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:37.738009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 563/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:37 2022] GET /identity/v3/projects/13058ee67b994090a55901ddc37db78e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:37.744043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:37.822818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:37.840151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0365-a9b9-411b-9f4e-087fdc4bced2 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:37.871755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda92c67-2b54-4a52-a24f-5c46f8d31d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:37.872647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda92c67-2b54-4a52-a24f-5c46f8d31d05 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:37.872926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda92c67-2b54-4a52-a24f-5c46f8d31d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:37.873199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda92c67-2b54-4a52-a24f-5c46f8d31d05 None admin] PATH_INFO: `/v3/projects/13058ee67b994090a55901ddc37db78e/users/a0039171b32648cf9664a9f4692e22f6/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:37.875365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda92c67-2b54-4a52-a24f-5c46f8d31d05 None admin] RBAC: Authorizing `identity:create_grant(project_id=13058ee67b994090a55901ddc37db78e, user_id=a0039171b32648cf9664a9f4692e22f6, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:37.876804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda92c67-2b54-4a52-a24f-5c46f8d31d05 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:37.986726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 590/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:06:37 2022] PUT /identity/v3/projects/13058ee67b994090a55901ddc37db78e/users/a0039171b32648cf9664a9f4692e22f6/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:06:40.187156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6c0d7-1cbf-470e-911f-ca4e28319c23 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:40.187723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6c0d7-1cbf-470e-911f-ca4e28319c23 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:40.187723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6c0d7-1cbf-470e-911f-ca4e28319c23 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:40.189115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 564/1154] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:40.194264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:40.194672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:40.194816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:40.211152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:40.212365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:40.276980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 591/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:40.301211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6c0d7-1cbf-470e-911f-ca4e28319c23 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:40.389586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:40.400391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6c0d7-1cbf-470e-911f-ca4e28319c23 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:40.434042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:40.434822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:40.435099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:40.435369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:40.445795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:40.453185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:40.454300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:40.455347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 565/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:40.461002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:40.529705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:40.557848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4e4f-0d48-42f3-bf04-5590b8defaf1 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:40.591902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:40.592600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:40.593054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:40.593752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:40.594497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:40.594810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:40.596185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:40.604828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 592/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:40.610598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:40.715236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e4f0-45db-488e-be4f-eac689174609 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:40.747893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:40.749043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:40.749322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:40.749595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] PATH_INFO: `/v3/users/a0039171b32648cf9664a9f4692e22f6` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:40.750767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] RBAC: Authorizing `identity:get_user(user_id=a0039171b32648cf9664a9f4692e22f6)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:40.752197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:40.753831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 566/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:40 2022] GET /identity/v3/users/a0039171b32648cf9664a9f4692e22f6 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:40.759610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:40.858449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1ed51-df5b-45f8-9c3d-d21f6459d6ab None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:40.890992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:40.891628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:40.891907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:40.892178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:40.895403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:40.957965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:40.959433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:41.054924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 593/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:41.060683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:41.082048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:41.156647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5de995-6459-42cc-8ed4-01e91f0c0341 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:41.189608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded1edb-c7e7-4738-9ab3-294473d7d16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:41.190455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1edb-c7e7-4738-9ab3-294473d7d16c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:41.190767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1edb-c7e7-4738-9ab3-294473d7d16c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:41.191035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1edb-c7e7-4738-9ab3-294473d7d16c None admin] PATH_INFO: `/v3/projects/13058ee67b994090a55901ddc37db78e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:41.192197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded1edb-c7e7-4738-9ab3-294473d7d16c None admin] RBAC: Authorizing `identity:get_project(project_id=13058ee67b994090a55901ddc37db78e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:41.193645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded1edb-c7e7-4738-9ab3-294473d7d16c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:41.195350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 567/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:41 2022] GET /identity/v3/projects/13058ee67b994090a55901ddc37db78e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:41.201421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:41.234641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:41.299461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087587e9-872d-4ffc-bd6a-eff546a4ecaa None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:41.332881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:41.333457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:41.333740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:41.334015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:41.334833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=13058ee67b994090a55901ddc37db78e, user.id=a0039171b32648cf9664a9f4692e22f6) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:41.335211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:41.336657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e3e2-2e28-44e7-9258-cbc8dcdd4a4d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:41.342342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 594/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:41 2022] GET /identity/v3/role_assignments?user.id=a0039171b32648cf9664a9f4692e22f6&scope.project.id=13058ee67b994090a55901ddc37db78e&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:43.585874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e7c57-ba6c-4072-bba2-187c8b53d594 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:43.586403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e7c57-ba6c-4072-bba2-187c8b53d594 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:43.586403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e7c57-ba6c-4072-bba2-187c8b53d594 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:43.587713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 568/1162] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:43.593162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:43.593603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:43.593720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:43.611078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:43.612265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:43.680596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 595/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:43.703192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e7c57-ba6c-4072-bba2-187c8b53d594 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:43.791610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:43.804009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e7c57-ba6c-4072-bba2-187c8b53d594 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:43.837849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:43.838418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:43.838702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:43.838987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:43.904506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:43.905954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:43.989207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:44.001517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 569/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:44.008107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:44.037215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:44.106034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9060cf-ad24-4650-ab2d-bb451544dcd3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:44.138783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:44.139573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:44.139856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:44.140132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:44.147179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:44.148629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d89fd5de-e068-4c99-b1bf-4de31215480c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:44.162402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:44.163600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:06:44.164752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 596/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:06:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:44.169578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:44.265143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cf3aa-0998-49ec-aae4-62526ae36c40 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:44.298714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:44.299374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:44.299657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:44.299930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:44.301225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:44.301794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:44.302878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25197d-7cbd-4bf5-89a6-a5b9545af195 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:44.310090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 570/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Sat Jun 4 09:06:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:46.523861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926e76a-a2a5-40db-a2bc-c7ee5036e46a None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:46.524480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926e76a-a2a5-40db-a2bc-c7ee5036e46a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:46.524480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926e76a-a2a5-40db-a2bc-c7ee5036e46a None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:46.525726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 597/1167] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:46.530890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579e3a-9bd9-4b0d-9860-4df635ce0e6a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:46.531269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579e3a-9bd9-4b0d-9860-4df635ce0e6a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:46.531447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579e3a-9bd9-4b0d-9860-4df635ce0e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:46.547919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c3579e3a-9bd9-4b0d-9860-4df635ce0e6a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:46.549134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c3579e3a-9bd9-4b0d-9860-4df635ce0e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:46.595128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:46.615007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 571/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:46.639167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926e76a-a2a5-40db-a2bc-c7ee5036e46a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:46.728040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:46.741130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926e76a-a2a5-40db-a2bc-c7ee5036e46a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:46.775053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12021e5d-887f-4e0b-9fd8-e36a1b5174f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:46.775631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12021e5d-887f-4e0b-9fd8-e36a1b5174f6 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:46.775911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12021e5d-887f-4e0b-9fd8-e36a1b5174f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:46.776185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12021e5d-887f-4e0b-9fd8-e36a1b5174f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:46.776956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12021e5d-887f-4e0b-9fd8-e36a1b5174f6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:46.778322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12021e5d-887f-4e0b-9fd8-e36a1b5174f6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:46.854256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 598/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Sat Jun 4 09:06:46 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:49.094123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800f7f0-986d-4e3f-837e-66b95c0e59ea None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:49.094692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800f7f0-986d-4e3f-837e-66b95c0e59ea None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:49.094692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800f7f0-986d-4e3f-837e-66b95c0e59ea None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:49.096070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 572/1170] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:49.101800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:49.102187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:49.102354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:49.119281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:49.120483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:49.184854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 599/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:49.208229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800f7f0-986d-4e3f-837e-66b95c0e59ea None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:49.307656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800f7f0-986d-4e3f-837e-66b95c0e59ea None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:49.340633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7662fb9-3a35-4e1b-8805-4729e70f3344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:49.341482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7662fb9-3a35-4e1b-8805-4729e70f3344 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:49.341761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7662fb9-3a35-4e1b-8805-4729e70f3344 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:49.342035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7662fb9-3a35-4e1b-8805-4729e70f3344 None admin] PATH_INFO: `/v3/domains/840fcc2708254ecbadfaf6756ddd130c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:49.352904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7662fb9-3a35-4e1b-8805-4729e70f3344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=840fcc2708254ecbadfaf6756ddd130c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:49.354283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7662fb9-3a35-4e1b-8805-4729e70f3344 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:49.355937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 573/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Sat Jun 4 09:06:49 2022] GET /identity/v3/domains/840fcc2708254ecbadfaf6756ddd130c => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:49.361936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:49.438116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:49.459111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355105f-7e0b-453a-8f77-1520558700c4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:49.492850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3f808-023f-471b-8c8a-8c4570291096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:49.493506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3f808-023f-471b-8c8a-8c4570291096 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:49.493790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3f808-023f-471b-8c8a-8c4570291096 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:49.494069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3f808-023f-471b-8c8a-8c4570291096 None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:49.494772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3f808-023f-471b-8c8a-8c4570291096 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:49.496212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3f808-023f-471b-8c8a-8c4570291096 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:49.613544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 600/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:06:49 2022] POST /identity/v3/projects => generated 361 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:51.877596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bf7e6-5221-40d9-a9f0-1699a98171dc None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:51.878189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bf7e6-5221-40d9-a9f0-1699a98171dc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:51.878189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bf7e6-5221-40d9-a9f0-1699a98171dc None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:51.880119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 574/1174] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:51.885521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:51.885926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:51.886078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:51.902527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:51.903709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:51.968448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 601/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:51.993401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bf7e6-5221-40d9-a9f0-1699a98171dc None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:52.037597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:52.093278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bf7e6-5221-40d9-a9f0-1699a98171dc None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:52.126709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd042-7dcf-48a7-b187-cf5ba3551205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:52.127564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd042-7dcf-48a7-b187-cf5ba3551205 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:52.127892 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd042-7dcf-48a7-b187-cf5ba3551205 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:52.128116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd042-7dcf-48a7-b187-cf5ba3551205 None admin] PATH_INFO: `/v3/domains/840fcc2708254ecbadfaf6756ddd130c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:52.129342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd042-7dcf-48a7-b187-cf5ba3551205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=840fcc2708254ecbadfaf6756ddd130c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:52.130775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd042-7dcf-48a7-b187-cf5ba3551205 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:52.132406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 575/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Sat Jun 4 09:06:51 2022] GET /identity/v3/domains/840fcc2708254ecbadfaf6756ddd130c => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:52.138123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:52.195396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:52.234799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdf618-cdbc-43e9-9e3a-6d8c26d26f7e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:52.271678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bd6de-8f37-4bf9-8cf5-a88465d2204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:52.272395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bd6de-8f37-4bf9-8cf5-a88465d2204c None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:52.272681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bd6de-8f37-4bf9-8cf5-a88465d2204c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:52.272960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bd6de-8f37-4bf9-8cf5-a88465d2204c None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:52.273672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bd6de-8f37-4bf9-8cf5-a88465d2204c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:52.275117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bd6de-8f37-4bf9-8cf5-a88465d2204c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:52.365108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 602/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:06:52 2022] POST /identity/v3/users => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:06:54.631116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414836-0e54-49d4-bcc1-e38d83fe922f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:54.631625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414836-0e54-49d4-bcc1-e38d83fe922f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:54.631625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414836-0e54-49d4-bcc1-e38d83fe922f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:54.632989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 576/1178] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:54.638622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:54.639067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:54.639157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:54.656358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:54.657560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:54.721640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 603/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:54.745327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb414836-0e54-49d4-bcc1-e38d83fe922f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:54.847109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb414836-0e54-49d4-bcc1-e38d83fe922f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:54.857487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:54.881347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:54.882074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:54.882358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:54.882635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:54.892535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:54.901014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:54.902154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:54.903359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 577/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:54.908224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:54.954140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:55.005971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f646cc0-f4d3-4c0d-b33d-9bd183eb933c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:55.039880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:55.040574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:55.040865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:55.041132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:55.041870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:55.042178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:55.043543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:55.052173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 604/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:55.057999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:55.155369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e51be-c65e-4f0e-8aeb-b14a0b46c2d5 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:55.189156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:55.190019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:55.190300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:55.190575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] PATH_INFO: `/v3/users/1e28cb4e2ee747b0b6e9932c51bf5a40` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:55.204166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] RBAC: Authorizing `identity:get_user(user_id=1e28cb4e2ee747b0b6e9932c51bf5a40)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:55.205656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:55.207316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 578/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:55 2022] GET /identity/v3/users/1e28cb4e2ee747b0b6e9932c51bf5a40 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:55.213147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:55.311227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ca16-6370-40ed-b992-41ad6b212d8f None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:55.320092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:55.344002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:55.344678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:55.344966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:55.345244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:55.412143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:55.413553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:55.506992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 605/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:06:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:06:55.512993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:55.542453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:55.611601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd6d71-440c-413e-90e5-f83d49c16b8a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:55.644650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcc42a-67ef-431a-a3ac-3bcdb3a399a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:55.645503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc42a-67ef-431a-a3ac-3bcdb3a399a0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:55.646223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc42a-67ef-431a-a3ac-3bcdb3a399a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:55.646223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc42a-67ef-431a-a3ac-3bcdb3a399a0 None admin] PATH_INFO: `/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:55.647223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcc42a-67ef-431a-a3ac-3bcdb3a399a0 None admin] RBAC: Authorizing `identity:get_project(project_id=dbcc74dbf2914a54ab9b8ccc227ed811)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:55.648681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcc42a-67ef-431a-a3ac-3bcdb3a399a0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:55.650381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 579/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:55 2022] GET /identity/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:55.656746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:55.667573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:55.752933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2e3a-22bc-483c-b751-269933dcf9b2 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:55.786235 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:55.786881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:55.787168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:55.787973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:55.788268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=dbcc74dbf2914a54ab9b8ccc227ed811, user.id=1e28cb4e2ee747b0b6e9932c51bf5a40) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:55.788613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:55.790037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d3e65-4516-4a4f-bdb0-b6a25336649a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:55.795407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 606/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:06:55 2022] GET /identity/v3/role_assignments?user.id=1e28cb4e2ee747b0b6e9932c51bf5a40&scope.project.id=dbcc74dbf2914a54ab9b8ccc227ed811&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:58.020099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54db8-4aa8-4a38-a00f-2a8d076c5b59 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.020707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54db8-4aa8-4a38-a00f-2a8d076c5b59 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.020707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54db8-4aa8-4a38-a00f-2a8d076c5b59 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.022059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 580/1186] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:06:58.027325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.027777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.027882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.045069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:06:58.046295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:06:58.111822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 607/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:06:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:06:58.135015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54db8-4aa8-4a38-a00f-2a8d076c5b59 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:58.231032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:58.238281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54db8-4aa8-4a38-a00f-2a8d076c5b59 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:58.272523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:58.273605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.273894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.274458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.284743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:06:58.292648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:06:58.293793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:06:58.294711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 581/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:06:58.299836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:58.302047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:58.411542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd2a98-fc21-403f-b842-a38dfde4b342 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:58.448556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:58.449280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.449562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.449837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.450580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:06:58.450886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:58.452258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:58.460754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 608/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:58.466807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:58.568164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e0bba-385b-4619-946d-38916205e3f3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:58.600887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:58.601760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.602044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.602320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] PATH_INFO: `/v3/users/1e28cb4e2ee747b0b6e9932c51bf5a40` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.603517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] RBAC: Authorizing `identity:get_user(user_id=1e28cb4e2ee747b0b6e9932c51bf5a40)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:58.604981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:58.606629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 582/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:06:58 2022] GET /identity/v3/users/1e28cb4e2ee747b0b6e9932c51bf5a40 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:58.612554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:58.674899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:58.711759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ef8a-d94f-4e78-98e3-b663e9232436 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:58.744429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df90e9e-0578-443f-88f1-90dbb46b9e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:58.745281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df90e9e-0578-443f-88f1-90dbb46b9e61 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.745592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df90e9e-0578-443f-88f1-90dbb46b9e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.745865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df90e9e-0578-443f-88f1-90dbb46b9e61 None admin] PATH_INFO: `/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.746982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df90e9e-0578-443f-88f1-90dbb46b9e61 None admin] RBAC: Authorizing `identity:get_project(project_id=dbcc74dbf2914a54ab9b8ccc227ed811)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:58.748472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df90e9e-0578-443f-88f1-90dbb46b9e61 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:58.750404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 609/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:06:58 2022] GET /identity/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:06:58.757010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:06:58.782209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:06:58.856647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf49af-b753-4d24-b89e-8ea41cc6d1f8 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:06:58.890696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbfd92-a9ad-4b46-abb3-5fc1e3f6be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:06:58.891585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbfd92-a9ad-4b46-abb3-5fc1e3f6be2d None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:06:58.891914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbfd92-a9ad-4b46-abb3-5fc1e3f6be2d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:06:58.892193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbfd92-a9ad-4b46-abb3-5fc1e3f6be2d None admin] PATH_INFO: `/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811/users/1e28cb4e2ee747b0b6e9932c51bf5a40/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:06:58.894176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbfd92-a9ad-4b46-abb3-5fc1e3f6be2d None admin] RBAC: Authorizing `identity:create_grant(project_id=dbcc74dbf2914a54ab9b8ccc227ed811, user_id=1e28cb4e2ee747b0b6e9932c51bf5a40, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:06:58.895635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbfd92-a9ad-4b46-abb3-5fc1e3f6be2d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:06:58.962014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 583/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:06:58 2022] PUT /identity/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811/users/1e28cb4e2ee747b0b6e9932c51bf5a40/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:07:01.194533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397da51-518d-41ff-ba46-8fc43c47566d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:01.195068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397da51-518d-41ff-ba46-8fc43c47566d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:01.195068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397da51-518d-41ff-ba46-8fc43c47566d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:01.196453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 610/1193] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:07:01.201705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:01.202103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:01.202269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:01.218615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:07:01.219801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:07:01.285578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 584/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:07:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:07:01.309522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397da51-518d-41ff-ba46-8fc43c47566d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:07:01.407481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397da51-518d-41ff-ba46-8fc43c47566d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:07:01.435479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:07:01.440104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:07:01.440910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:01.441190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:01.441461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:01.451637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:07:01.459581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:07:01.460711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:07:01.466572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 611/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:07:01.467449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:07:01.482221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:07:01.564252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa606f09-dd13-41b2-91a1-3854a593dc4c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:07:01.599551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:07:01.600216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:01.600519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:01.600788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:01.601527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:07:01.601827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:07:01.603198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:07:01.611397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 585/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:07:01.617204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:07:01.718840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db67807-0205-485a-9d0d-3a886ccdbba4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:07:01.751827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:07:01.752704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:01.752988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:01.753255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] PATH_INFO: `/v3/users/1e28cb4e2ee747b0b6e9932c51bf5a40` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:01.754448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] RBAC: Authorizing `identity:get_user(user_id=1e28cb4e2ee747b0b6e9932c51bf5a40)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:07:01.755861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:07:01.757502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 612/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Sat Jun 4 09:07:01 2022] GET /identity/v3/users/1e28cb4e2ee747b0b6e9932c51bf5a40 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:07:01.763268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:07:01.842063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:07:01.864892 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a74bca-15ac-44f9-834c-3f901ab31ad4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:07:01.899139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:07:01.899782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:01.900066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:01.900370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:01.965516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:07:01.966964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:07:02.039329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:07:02.062346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 586/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:07:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:07:02.068864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:07:02.126045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:07:02.165683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c064a1-7f8f-4a23-a6f6-89a022c649e0 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:07:02.199715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf88191-bc94-4cf8-87a6-c67939f62c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:07:02.200592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf88191-bc94-4cf8-87a6-c67939f62c0c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:02.200880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf88191-bc94-4cf8-87a6-c67939f62c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:02.201156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf88191-bc94-4cf8-87a6-c67939f62c0c None admin] PATH_INFO: `/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:02.202344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf88191-bc94-4cf8-87a6-c67939f62c0c None admin] RBAC: Authorizing `identity:get_project(project_id=dbcc74dbf2914a54ab9b8ccc227ed811)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:07:02.203772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf88191-bc94-4cf8-87a6-c67939f62c0c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:07:02.205702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 613/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Sat Jun 4 09:07:02 2022] GET /identity/v3/projects/dbcc74dbf2914a54ab9b8ccc227ed811 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:07:02.212026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:07:02.312268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8572cf1-88a3-4bb9-92da-46c74187e188 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:07:02.345487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:07:02.346128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:07:02.346402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:07:02.346674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:07:02.347544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=dbcc74dbf2914a54ab9b8ccc227ed811, user.id=1e28cb4e2ee747b0b6e9932c51bf5a40) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:07:02.347810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:07:02.349335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78247-11bf-407e-9974-56b17aff5e54 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:07:02.355108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 587/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:07:02 2022] GET /identity/v3/role_assignments?user.id=1e28cb4e2ee747b0b6e9932c51bf5a40&scope.project.id=dbcc74dbf2914a54ab9b8ccc227ed811&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:07:12.195423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=36 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:07:12.352489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=80 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:21.537628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0260cbc-5c0d-45fa-be78-b9bb032c0268 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:21.538353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0260cbc-5c0d-45fa-be78-b9bb032c0268 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:21.538353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0260cbc-5c0d-45fa-be78-b9bb032c0268 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:21.539498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 614/1201] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:21.544947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:21.546299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:21.546299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:21.562158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:21.563369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:21.631006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 588/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:21.657001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0260cbc-5c0d-45fa-be78-b9bb032c0268 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:21.760185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0260cbc-5c0d-45fa-be78-b9bb032c0268 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:21.794225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:21.794931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:21.795217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:21.795489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:21.806207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:21.807632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:21.826715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:21.827914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:21.829032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:21.829032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:21.829032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:21.829032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:21.829032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 615/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:21.834196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:21.933643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e66187-fbef-4dc3-bd79-70eb13337f91 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:21.962391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:21.967893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:21.968596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:21.968881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:21.969164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:21.969881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:21.970188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:21.971582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eada4c-2f5e-42b6-bd7c-25b33fd84b10 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:21.983206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 589/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:21.989035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:22.043645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:22.090861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b48a4-a52b-4724-be1a-87db365665ac None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:22.125821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3d2fb-1246-4588-b584-d3a6eb215d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:22.126489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3d2fb-1246-4588-b584-d3a6eb215d0d None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:22.126770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3d2fb-1246-4588-b584-d3a6eb215d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:22.127043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3d2fb-1246-4588-b584-d3a6eb215d0d None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:22.128002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3d2fb-1246-4588-b584-d3a6eb215d0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:22.129487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3d2fb-1246-4588-b584-d3a6eb215d0d None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:22.213646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 616/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Sat Jun 4 09:08:21 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:08:24.521572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71e8a-502c-4d75-b44a-55a82cc4759c None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:24.522112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71e8a-502c-4d75-b44a-55a82cc4759c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:24.522112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71e8a-502c-4d75-b44a-55a82cc4759c None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:24.524129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 590/1206] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:24.529985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:24.530553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:24.530553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:24.548331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:24.549589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:24.622990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 617/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:24.648228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71e8a-502c-4d75-b44a-55a82cc4759c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:24.750286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71e8a-502c-4d75-b44a-55a82cc4759c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:24.785719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:24.786375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:24.786650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:24.786923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:24.797492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:08:24.806135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:24.807335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:24.808483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 591/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:08:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:24.813549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:24.852525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:24.910850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb9c88-6d9f-48b2-b373-f8f97298315b None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:24.945330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:24.945995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:24.950487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:24.950487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:24.950487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:24.950487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:24.950487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:24.957038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 618/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:08:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:24.962941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:25.015040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:25.065759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4b11d-a94f-4582-9ee8-5a8959ad2f4a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:25.100177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:25.101537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:25.101808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:25.102064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:25.112907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:25.114328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:25.132887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:25.134079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:25.135053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:25.135053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:25.135053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:25.135053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:25.135053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 592/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:25.140234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:25.240116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:25.243611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1911ca-2144-4045-bdd1-eaab5e6f1b2a None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:25.276803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:25.277467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:25.277749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:25.278026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:25.278741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:25.279049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:25.280453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:25.293173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 619/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:25.298725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:25.399692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37da9-d33a-4868-b568-cfcc76b289e6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:25.401236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:25.437781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:25.438652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:25.438940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:25.439272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:25.446756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:25.448200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:08:25.463747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:25.464987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:25.466197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 593/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:25.471005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:25.572974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee437941-12c7-41a5-8154-93f7135b7785 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:25.609885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:25.610583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:25.610886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:25.611139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:25.611999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:25.612335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:25.613814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:25.635525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 620/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Sat Jun 4 09:08:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:25.641687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:25.754044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9a0a7-b8d8-4ea2-91f5-1457ca25ac1b None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:25.794799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:25.795466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:25.795754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:25.796035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:25.830291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:25.870899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:25.872434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:25.980448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 594/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:08:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:08:25.986801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:26.011013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:26.094791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6037a-18d4-41b5-9ec2-c1bdea705bbd None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:26.129599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:26.130385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:26.130662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:26.130930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:26.141394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:26.142845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:26.160811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:26.161978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:26.162967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:26.162967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:26.162967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:26.162967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:26.162967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 621/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:26.167742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:26.228026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:26.266766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286de3b-7e1b-409c-9ccc-829d7891afb0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:26.302341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:26.303009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:26.303294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:26.304102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:26.304859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:26.305169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:26.306536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:26.318625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 595/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:26.324914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:26.394280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:26.424063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963e23-b652-48e8-8073-dacdb09d1677 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:26.457747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:26.458535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:26.458876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:26.459155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:26.465766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:26.467283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:08:26.480730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:26.481875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:26.490686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 622/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:26.491841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:26.587740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4868ed8-6fae-450a-9169-17fe3a74984a None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:26.610683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:26.624030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:26.624731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:26.625021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:26.625304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:26.626090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:26.626398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:26.628101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91218-dd32-484f-8121-b180dc4cae1a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:26.640188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 596/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:26.646005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:26.750478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def6446-0700-4934-87ea-763286c98f48 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:26.778621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:26.784009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:26.784710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:26.784990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:26.785266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:26.786094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=fa5724e78cc848a4af84a78bac03f64d) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:26.786404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:26.787828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af732a9-b0fd-42d4-a6be-32a10e4662c9 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:26.793673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 623/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:08:26 2022] GET /identity/v3/role_assignments?user.id=fa5724e78cc848a4af84a78bac03f64d&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:29.098732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663cf85-1846-46e3-8459-7d3c071b4cad None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:29.099302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663cf85-1846-46e3-8459-7d3c071b4cad None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:29.099302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663cf85-1846-46e3-8459-7d3c071b4cad None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:29.101224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 597/1220] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:29.106918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:29.109944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:29.109944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:29.125355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:29.127061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:29.208097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 624/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:29.232371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663cf85-1846-46e3-8459-7d3c071b4cad None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:29.345634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663cf85-1846-46e3-8459-7d3c071b4cad None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:29.383848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:29.384595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:29.384877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:29.385155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:29.396005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:08:29.404395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:29.406200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:29.407790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 598/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:08:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:29.411963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:29.523251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a9596-96a0-4b1d-ad11-de71d6f9e5ad None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:29.549465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:29.560858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:29.561550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:29.561958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:29.562282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:29.563065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:29.563367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:29.564826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:29.576764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 625/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:08:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:29.582544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:29.630923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:29.688675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca27a5-f18b-4899-829b-81ca76801ef4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:29.722748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:29.723540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:29.723824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:29.724102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:29.737870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:29.739399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:29.757923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:29.760397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:29.761527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:29.761527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:29.761527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:29.761527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:29.761527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 599/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:29.764967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:29.869386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecfd94-4b56-4bfc-b4f1-b24df25cd3ac None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:29.912473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:29.913135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:29.913417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:29.913701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:29.914417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:29.914726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:29.916095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:29.927855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 626/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:29.934240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:30.038354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:30.044868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea2d9-98e4-47bc-8a27-5fd91e679ddd None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:30.079009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:30.079816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:30.080100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:30.080410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:30.087512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:30.089031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:08:30.104772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:30.107240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:30.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 600/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:30.112009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:30.159839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:30.215104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b17d0-c882-4c23-8504-f9a436763233 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:30.249488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:30.250167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:30.250446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:30.250721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:30.251613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:30.251918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:30.253649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:30.272909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 627/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Sat Jun 4 09:08:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:30.279618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:30.390756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872798-1680-48fc-9ed3-1f2cec8ec8e4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:30.426291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:30.427092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:30.427381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:30.427655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:30.439383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:30.440906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:30.459364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:30.460611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:30.461530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:30.461530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:30.461530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:30.461530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:30.461530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 601/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:30.467134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:30.576617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5233e-1575-46ce-857b-d150f65d05d8 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:30.579777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:30.616187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:30.617438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:30.617726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:30.618008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:30.618733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:30.619040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:30.620524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:30.632565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 628/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:30.640114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:30.649885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:30.743207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027881c-a286-4705-853e-58357bec2ff7 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:30.778602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:30.779398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:30.779685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:30.779956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:30.786761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:30.788831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:08:30.803606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:30.809027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:30.810435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 602/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:30.810984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:30.919188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f1e2f-c3c6-47f5-9bc7-2cf361fa2cd3 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:30.954409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:30.958836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:30.958836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:30.958836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:30.958836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:30.958836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:30.958836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921587-7d1f-4c59-91f9-82e812d0871a None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:30.971906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 629/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:30.978883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:31.048797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:31.086762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b85b3-a246-40c7-98ee-adcebcb69fd3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:31.121441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f90d9-2259-4360-b8b7-7d3228342710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:31.122336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f90d9-2259-4360-b8b7-7d3228342710 None admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:31.122620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f90d9-2259-4360-b8b7-7d3228342710 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:31.122896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f90d9-2259-4360-b8b7-7d3228342710 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/fa5724e78cc848a4af84a78bac03f64d/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:31.140178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f90d9-2259-4360-b8b7-7d3228342710 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=fa5724e78cc848a4af84a78bac03f64d, role_id=5c7ecc913b4c49f8a8c6a0f2998bf5db)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:31.141680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f90d9-2259-4360-b8b7-7d3228342710 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:31.223760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 603/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:08:30 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/fa5724e78cc848a4af84a78bac03f64d/roles/5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:08:33.507745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f377c1b-9637-4769-a542-26b35b5470a4 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:33.508352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f377c1b-9637-4769-a542-26b35b5470a4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:33.508352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f377c1b-9637-4769-a542-26b35b5470a4 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:33.509685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 630/1233] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:08:33.515151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:33.515568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:33.515715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:33.532947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:33.534081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:33.607165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 604/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:33.631982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f377c1b-9637-4769-a542-26b35b5470a4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:33.659016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:33.738057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f377c1b-9637-4769-a542-26b35b5470a4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:33.783371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:33.784107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:33.784423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:33.784702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:33.796711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:08:33.805246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:33.806427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:33.812246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 631/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Sat Jun 4 09:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:33.813418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:33.862342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:33.921010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cafa4-e2fe-44f6-a70b-62f5e9153818 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:33.956098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:33.956791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:33.957077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:33.957357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:33.958106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] RBAC: Adding query filter params (name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:33.958416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:33.959796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:33.967829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 605/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Sat Jun 4 09:08:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:33.973540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:34.066971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:34.085902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73360a34-2f59-41cb-aa9d-6bdbfeaf359e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:34.127422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:34.128539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:34.128817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:34.129086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:34.142564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:34.144183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:34.163088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:34.164784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:34.166147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:34.166147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:34.166147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:34.166147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:34.166147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 632/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:34.170827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:34.277904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32057560-aa7a-408e-9a01-1f48d426088d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:34.287630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:34.313648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:34.314358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:34.314674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:34.314953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:34.315711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:34.316036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:34.320624 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:34.329592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 606/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:34.336700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:34.442865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f79791-c121-485e-b464-2ad4ecfa3406 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:34.478622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:34.479570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:34.479939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:34.480428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:34.483650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:34.489210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:34.490706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:08:34.506101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:34.507176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:34.508890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 633/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:34.513282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:34.617917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb3e2-9985-4c42-af17-631b698ad4a0 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:34.653290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:34.654016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:34.654306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:34.654582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:34.655568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:34.655747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:34.657393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:34.680212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 607/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Sat Jun 4 09:08:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:34.686503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:34.789100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375abf8-8f31-4013-a6f7-b0487a5e62ae None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:34.823306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:34.823975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:34.824254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:34.824556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:34.890901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:34.898506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:34.900037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:34.999481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 634/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:08:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:08:35.005604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:35.034916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:35.109808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7681-3801-4bee-8137-bb1e44df37cf None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:35.145673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:35.146464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:35.146740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:35.147011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:35.157277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:35.158734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:35.175795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:35.177001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:35.178010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:35.178010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:35.178010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:35.178010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:35.178010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 608/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:35.183244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:35.282063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:35.288369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df7bf7-7fc6-4f71-94e2-ce86e758fa59 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:35.325153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:35.325817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:35.326102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:35.326380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:35.327116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:35.327422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:35.328877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:35.341703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 635/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:35.347889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:35.432925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:35.454081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac6e56-d83c-4a5f-b924-cfa3674817db None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:35.487311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:35.488107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:35.488427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:35.488706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:35.496043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:35.497539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:08:35.512107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:35.513259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:35.514378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 609/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:35.519536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:35.621029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1693b-3bf1-42a5-98fa-221a411a3ff4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:35.655859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:35.656490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:35.656776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:35.657055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:35.657843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:35.658150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:35.659587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b8983-713e-4441-84a7-bb85941b5b5c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:35.672306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 636/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:35.678413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:35.777392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203009c-a778-4e50-b735-8838e4f8135f None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:35.811560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:35.812348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:35.812662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:35.812936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:35.813820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] RBAC: Adding query filter params (role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=fa5724e78cc848a4af84a78bac03f64d) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:35.814135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:35.815569 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9242-6035-4e42-b199-3360633d9d2a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:35.821937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 610/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:08:35 2022] GET /identity/v3/role_assignments?user.id=fa5724e78cc848a4af84a78bac03f64d&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=5c7ecc913b4c49f8a8c6a0f2998bf5db => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:38.172529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e62a7-1340-490e-8d27-6925c897a3fc None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:38.173221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e62a7-1340-490e-8d27-6925c897a3fc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:38.173221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e62a7-1340-490e-8d27-6925c897a3fc None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:38.174477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 637/1247] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:38.180728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:38.181165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:38.181276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:38.186384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:38.199214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:38.200443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:38.266657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 611/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:38.290383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e62a7-1340-490e-8d27-6925c897a3fc None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:38.309704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:38.397740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e62a7-1340-490e-8d27-6925c897a3fc None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:38.440610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:38.441333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:38.441607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:38.441881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:38.454750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:08:38.462724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:38.463959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:38.465158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 638/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:08:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:38.470870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:38.578417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22d08a-941b-432d-8fad-6c3e0c01704e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:38.608562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:38.613887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:38.614561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:38.614841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:38.615771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:38.615771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:38.616167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:38.618213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:38.627270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 612/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:08:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:38.633258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:38.713504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:38.737164 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7d5c-92ca-4031-b8b4-0032a8c0884b None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:38.771714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:38.772537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:38.772819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:38.773094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:38.783696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:38.785169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:38.802924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:38.807477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:38.808463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:38.808463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:38.808463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:38.808463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:38.808463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 639/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:38.810873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:38.919504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7208f-6dbb-46ed-abee-c1ed37090162 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:38.957557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:38.958348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:38.958670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:38.959016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:38.959796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:38.960743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:38.962300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:38.985767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 613/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:08:38.991507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:39.096511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021a56f-d0c8-4c86-ae8c-68961ab05050 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:39.123605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:39.132591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:39.133388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:39.133669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:39.133945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:39.143837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:39.145338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:08:39.160368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:39.161523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:39.162697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 640/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:39.168250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:39.224871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:39.273408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dfe75-d1b8-46f7-bb92-d6567bcda698 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:39.306955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:39.307622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:39.307900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:39.308172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:39.309039 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:39.309343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:39.310784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:39.329259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 614/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Sat Jun 4 09:08:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:39.335891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:39.436081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c66f3c-d904-4aa9-8200-ea13ade797bd None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:39.469880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:39.470513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:39.470794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:39.471065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:39.526748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:39.538585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:39.540026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:39.638976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 641/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:08:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:08:39.645265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:39.753872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426897f-4fcc-436d-b7dd-109f44028d65 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:39.767837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:39.788668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:39.789392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:39.789667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:39.789939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:39.800059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:39.801534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:39.819223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:39.824144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:39.825443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:39.825443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:39.825443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:39.825443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:39.825443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 615/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:39.826994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:39.913888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:39.925854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6486a-db40-4f74-b85c-bd6609761bd4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:39.960144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:39.961048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:39.961337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:39.961621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:39.962344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:39.962664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:39.964038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:39.975260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 642/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:39.981152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:40.087301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97a20-cc59-41f1-b036-8d0d2c398cb4 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:40.123347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:40.124147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:40.124459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:40.124733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:40.132185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:40.133683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:08:40.147975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:40.149127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:40.150365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 616/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:40.155229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:40.260356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2fcc2-587e-4073-a5b9-9d366b7f187e None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:40.296672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:40.297340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:40.297626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:40.297904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:40.298692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:40.299012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:40.300500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b76891-708a-4872-b2e5-8da98f336d5c None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:40.314095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 643/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:40.320204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:40.346916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:40.423939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a692bd-479e-4084-890e-f81151fc2417 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:40.458304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:40.458960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:40.459246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:40.459518 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:40.460902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=fa5724e78cc848a4af84a78bac03f64d) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:40.461215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:40.462657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2553a5c-ca3c-432f-b90b-f3b7b40de91a None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:40.468296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 617/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:08:40 2022] GET /identity/v3/role_assignments?user.id=fa5724e78cc848a4af84a78bac03f64d&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:42.746242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ffe4-71d2-4732-b782-f83193e422b9 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:42.746845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ffe4-71d2-4732-b782-f83193e422b9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:42.746845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ffe4-71d2-4732-b782-f83193e422b9 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:42.748171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 644/1261] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:42.753953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:42.754341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:42.754520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:42.771913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:42.773186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:42.840171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 618/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:42.863433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2ffe4-71d2-4732-b782-f83193e422b9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:42.874965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:42.963799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2ffe4-71d2-4732-b782-f83193e422b9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:42.998387 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:42.999101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:42.999386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:42.999657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.010691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:08:43.018300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:43.019406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:43.020487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 645/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:43.025729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:43.044525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:43.126248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556180a-f936-489e-b74b-1086b21a3e87 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:43.161146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:43.161813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:43.162094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:43.162373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.163121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:43.163963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:43.165401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:43.174365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 619/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:08:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:43.181256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:43.250791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:43.286046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a269d-e8d3-4965-8f4c-c813705e92d3 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:43.322122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:43.322926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:43.323209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:43.323480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.337032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:43.338513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:43.356416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:43.358126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:43.359964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:43.359964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:43.359964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:43.359964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:43.359964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 646/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:43.363471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:43.446762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:43.465940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808024-1e3a-4f2e-827f-f63e6f4bc2ac None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:43.500106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:43.500858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:43.501089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:43.501359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.502071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:43.502371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:43.503985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:43.515230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 620/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:43.520974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:43.622163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f12af-026d-4e36-b71f-4888af6cced3 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:43.641910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:43.656217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:43.657046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:43.657325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:43.657595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.665771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:43.667210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:08:43.681801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:43.683071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:43.684392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 647/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:43.688756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:43.787189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31600-4e33-419d-9ab0-87fff6ba3fe3 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:43.819491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:43.820725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:43.821003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:43.821274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.822099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:43.827971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:43.827971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:43.833066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:43.841982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 621/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Sat Jun 4 09:08:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:08:43.847611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:43.945868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721f1c3-9275-495d-9f19-9bf271cd6fbf None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:43.978883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:43.979667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:43.979954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:43.980225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:43.990507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:43.991995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:44.009913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:44.010867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:44.011957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:44.011957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:44.011957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:44.011957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:44.011957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 648/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:44.017031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:44.119308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bd9d0-284b-45d4-9b6e-a7698a5ea4f1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:44.158503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:44.159170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:44.159452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:44.159725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:44.160561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:44.160869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:44.162304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:44.174368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 622/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:44.180597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:44.213184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:44.281144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903f24-2863-477f-bbdc-f3e344647553 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:44.315831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:44.316655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:44.316941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:44.317232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:44.324916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:44.326373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:08:44.341414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:44.343957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:44.344956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 649/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:08:44.348473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:44.408902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:44.457113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b5648-eed3-4358-8968-7149c7227fc6 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:44.489795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:44.490433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:44.490712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:44.490982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:44.492262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:44.492591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:44.494017 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70c696-e4e8-4f5e-8624-940088751278 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:44.506703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 623/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:44.513449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:44.604467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:44.612896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6cc5f-4a9b-4097-8141-fea147a1c6f4 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:44.647461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8f52f-2f53-4867-9fe1-4f069ebbd799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:44.648385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8f52f-2f53-4867-9fe1-4f069ebbd799 None admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:44.648662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8f52f-2f53-4867-9fe1-4f069ebbd799 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:44.648933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8f52f-2f53-4867-9fe1-4f069ebbd799 None admin] PATH_INFO: `/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/fa5724e78cc848a4af84a78bac03f64d/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:44.651932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8f52f-2f53-4867-9fe1-4f069ebbd799 None admin] RBAC: Authorizing `identity:create_grant(project_id=625af3032fbf42baa92a110d3363c3e9, user_id=fa5724e78cc848a4af84a78bac03f64d, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:44.653800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8f52f-2f53-4867-9fe1-4f069ebbd799 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:44.782783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 650/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Sat Jun 4 09:08:44 2022] PUT /identity/v3/projects/625af3032fbf42baa92a110d3363c3e9/users/fa5724e78cc848a4af84a78bac03f64d/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:08:47.052974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c568-bc9c-4052-be74-dbdfd2ef38be None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.053555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c568-bc9c-4052-be74-dbdfd2ef38be None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.053555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c568-bc9c-4052-be74-dbdfd2ef38be None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.054928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 624/1274] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:47.060728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.061174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.061280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.078283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:47.079494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:47.150486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 651/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:47.174181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7c568-bc9c-4052-be74-dbdfd2ef38be None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:47.279027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7c568-bc9c-4052-be74-dbdfd2ef38be None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:47.297494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:47.315148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:47.316014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.316326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.316611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.327485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 04 09:08:47.335246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:47.336423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:47.337364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 625/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Sat Jun 4 09:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:47.342958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:47.428543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:47.447432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099756d-d64f-4845-aad3-d7521636ed44 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:47.480240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:47.480932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.481208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.481483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.482218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:47.482526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:47.483892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:47.492662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 652/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Sat Jun 4 09:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:47.498457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:47.601805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c6bba-2266-494d-8a5b-d34aa62217d2 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:47.639629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:47.640515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.640867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.641135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.652875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:47.654324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:47.672254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:47.673381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:47.674529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:47.674529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:47.674529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:47.674529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:47.674529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 626/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:47.680627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:47.780159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fddf4-0e61-4d29-9e43-bf6bb4e77c29 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:47.813398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:47.814049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.814376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.814654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.815363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:47.815665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:47.817051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:47.828896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 653/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:47.836473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:47.868059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:47.938217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94d4a-d3ab-4ff1-acc8-6b6cace25656 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:47.974085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:47.974887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:47.975174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:47.975460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:47.983560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:47.985094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 04 09:08:47.999784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:48.000989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:48.002138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 627/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:48.007192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:48.011067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:48.111461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bd778-c34c-4ea2-9cbf-5034fc6ac680 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:48.146964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:48.147673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:48.148561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:48.148844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:48.149698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:48.150006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:48.151482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:48.171034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 654/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Sat Jun 4 09:08:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:48.176887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:48.267654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:48.277812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315e68-7d4d-4d15-8358-63920f230c77 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:48.311357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:48.312002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:48.312299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:48.312585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:48.378439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:48.379924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:48.470608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:48.481410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 628/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Sat Jun 4 09:08:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 04 09:08:48.487361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:48.552368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:48.591509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f54ac-6523-4320-9c8e-6fd8a020b715 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:48.625352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:48.626166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:48.626447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:48.626728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:48.638010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:48.639477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 04 09:08:48.658178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:48.659342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:48.660469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 04 09:08:48.660469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 04 09:08:48.660469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 04 09:08:48.660469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:48.660469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 655/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:48.665590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:48.766864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30fad0-af02-4ed1-8959-3de8da3bded1 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:48.799184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:48.806805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:48.806805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:48.806805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:48.806805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:48.806805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:48.806805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:48.814829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 629/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Sat Jun 4 09:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:48.821341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:48.923697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c41f4-6938-4c86-a328-d124a236ed60 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:48.938879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:48.959267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:48.960052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:48.960367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:48.960649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:48.967355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:48.968979 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 04 09:08:48.983018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:48.984096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: ERROR keystone.server.flask.application Jun 04 09:08:48.985228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 656/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Sat Jun 4 09:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 04 09:08:48.990132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:49.042124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:49.093127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf6315-bee6-42b3-ad86-2b6416314f5d None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:49.127208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:49.127906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:49.128451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:49.128730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:49.129524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:49.129827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:49.131265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189fc2f-9d73-4121-b280-509fe2d8de75 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:49.143214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 630/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Sat Jun 4 09:08:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:49.149392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:49.248859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e30b3-0cc3-4055-8aa7-4d50de31a204 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:49.282494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:49.283137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:49.283420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:49.283688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:49.284542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] RBAC: Adding query filter params (role.id=c20defb0ae0a4e17b47a959da21cf17e, scope.project.id=625af3032fbf42baa92a110d3363c3e9, user.id=fa5724e78cc848a4af84a78bac03f64d) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:49.284846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:49.286266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b7ae5-5eb7-4a3b-be9e-f1c7b9c903b5 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:49.295331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 657/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Sat Jun 4 09:08:49 2022] GET /identity/v3/role_assignments?user.id=fa5724e78cc848a4af84a78bac03f64d&scope.project.id=625af3032fbf42baa92a110d3363c3e9&role.id=c20defb0ae0a4e17b47a959da21cf17e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:51.570775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79291973-85ed-40b5-a37b-4c3b9806d87b None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:51.571343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79291973-85ed-40b5-a37b-4c3b9806d87b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:51.571343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79291973-85ed-40b5-a37b-4c3b9806d87b None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:51.572699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 631/1288] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:51.579006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:51.579587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:51.579587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:51.598171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:51.599410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:51.635816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:51.682314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 658/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:51.706105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79291973-85ed-40b5-a37b-4c3b9806d87b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:51.821528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79291973-85ed-40b5-a37b-4c3b9806d87b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:51.834325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:51.858945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:51.859765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:51.860052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:51.860361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:51.861035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:51.862451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:51.875969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:51.877187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:51.878076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 632/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:08:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:51.884439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:51.996965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55720ec5-ab1a-4b16-bc2f-6dab6e6b493e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:52.034063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:52.034736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:52.035082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:52.035295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:52.036007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:52.036341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:52.041376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e1da7-abec-42e5-a24a-9c0aed95a66b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:52.043573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 659/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Sat Jun 4 09:08:51 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:52.049744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:52.148224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c357f39-4f99-4a93-95e9-4c1df186c2ad None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:52.188316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:52.189425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:52.189785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:52.190065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] PATH_INFO: `/v3/services` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:52.191104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:52.191414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:52.192803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a117b-cd0c-42fb-8211-81c237296637 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:52.197873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 633/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Sat Jun 4 09:08:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:54.439258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b4bd3-e1b3-4350-ad56-def19a6a7fdb None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:54.439834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b4bd3-e1b3-4350-ad56-def19a6a7fdb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:54.439945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b4bd3-e1b3-4350-ad56-def19a6a7fdb None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:54.441287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 660/1293] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:54.447305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f87756-c270-4532-a207-cbe867febed9 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:54.447760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f87756-c270-4532-a207-cbe867febed9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:54.447897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f87756-c270-4532-a207-cbe867febed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:54.465070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-41f87756-c270-4532-a207-cbe867febed9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:54.468181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-41f87756-c270-4532-a207-cbe867febed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:54.496953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:54.534291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 634/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:54.556886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b4bd3-e1b3-4350-ad56-def19a6a7fdb None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:54.588894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:54.659415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b4bd3-e1b3-4350-ad56-def19a6a7fdb None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:54.693264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dda2b5-25df-448d-943d-ed0a47110bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:54.693914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dda2b5-25df-448d-943d-ed0a47110bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:54.694190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dda2b5-25df-448d-943d-ed0a47110bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:54.694459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dda2b5-25df-448d-943d-ed0a47110bd3 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:54.695772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dda2b5-25df-448d-943d-ed0a47110bd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:54.697146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dda2b5-25df-448d-943d-ed0a47110bd3 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:54.809432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 661/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Sat Jun 4 09:08:54 2022] POST /identity/v3/services => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:08:57.088600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc64751-6d35-45c7-b9a7-4ba063724b79 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:57.089254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc64751-6d35-45c7-b9a7-4ba063724b79 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:57.089254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc64751-6d35-45c7-b9a7-4ba063724b79 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:57.090517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 635/1296] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:08:57.095924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:57.096427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:57.096603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:57.114194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:08:57.115408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:08:57.189258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 662/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:08:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:08:57.212851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc64751-6d35-45c7-b9a7-4ba063724b79 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:57.329764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc64751-6d35-45c7-b9a7-4ba063724b79 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:57.364625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:57.365348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:57.365631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:57.365904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:57.366555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:57.367958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:08:57.380258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:08:57.381731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:08:57.383234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:08:57.384637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 636/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:08:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:08:57.388342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:57.407375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:57.489935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84014-a48e-432b-84d4-1fef8ead4a06 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:57.530640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:57.531295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:57.531577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:57.531852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:57.532590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:57.532902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:57.534269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:57.539466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 663/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Sat Jun 4 09:08:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:08:57.545694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:57.578267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:57.650923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206ae74-acab-4ea1-8c81-f551617786fc None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:57.687136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:57.687795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:57.688078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:57.688388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:57.689183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] RBAC: Adding query filter params (interface=public, service_id=b2a645afcfd643cbbee9f2d4b7255c4f, region_id=RegionOne) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:08:57.689493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:57.690915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9913d-f81a-492e-93e9-61b6f04dd91c None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:57.696921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 637/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Sat Jun 4 09:08:57 2022] GET /identity/v3/endpoints?service_id=b2a645afcfd643cbbee9f2d4b7255c4f&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:08:57.703430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:08:57.811098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:08:57.830705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101997-548e-4a18-9aa0-0e61e9316d59 None admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:08:57.866340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56094544-768f-414b-a42d-3f5f24aa72c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:08:57.867534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56094544-768f-414b-a42d-3f5f24aa72c6 None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:08:57.869689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56094544-768f-414b-a42d-3f5f24aa72c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:08:57.869689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56094544-768f-414b-a42d-3f5f24aa72c6 None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:08:57.869689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56094544-768f-414b-a42d-3f5f24aa72c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:08:57.870183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56094544-768f-414b-a42d-3f5f24aa72c6 None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:08:57.877180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 664/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Sat Jun 4 09:08:57 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:09:00.160528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce63d7d-267e-4004-adeb-46be3159e34e None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:09:00.161119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce63d7d-267e-4004-adeb-46be3159e34e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:09:00.161273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce63d7d-267e-4004-adeb-46be3159e34e None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:09:00.162775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 638/1302] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:09:00.169104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:09:00.169588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:09:00.169734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:09:00.186319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:09:00.187524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:09:00.255658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 665/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:09:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 04 09:09:00.280184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce63d7d-267e-4004-adeb-46be3159e34e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:09:00.379781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:09:00.386503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce63d7d-267e-4004-adeb-46be3159e34e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:09:00.423417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:09:00.424144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:09:00.424456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:09:00.424732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:09:00.425382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:09:00.426759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 04 09:09:00.439059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 04 09:09:00.440271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.get_or_create( Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application with Lock( Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self._enter() Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application return self.creator() Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application created_value = creator( Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: ERROR keystone.server.flask.application Jun 04 09:09:00.445861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 639/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Sat Jun 4 09:09:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 04 09:09:00.446789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:09:00.529222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:09:00.556067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe6237-f0dc-4b43-99e0-a8574c632802 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:09:00.612493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:09:00.613173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:09:00.613452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:09:00.613722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] PATH_INFO: `/v3/services` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:09:00.614428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:09:00.614729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:09:00.616200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8257a-3828-4616-bdf1-250c1141841b None admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:09:00.622360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 666/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Sat Jun 4 09:09:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:09:00.628130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:09:00.732777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d8b7-0fc2-4c8d-ad08-e78755e6aaa9 None admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:09:00.770425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcecdc1-4047-42fd-8783-c99a64b39951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:09:00.771083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcecdc1-4047-42fd-8783-c99a64b39951 None admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:09:00.771365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcecdc1-4047-42fd-8783-c99a64b39951 None admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:09:00.771642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcecdc1-4047-42fd-8783-c99a64b39951 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:09:00.772426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcecdc1-4047-42fd-8783-c99a64b39951 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:09:00.774311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcecdc1-4047-42fd-8783-c99a64b39951 None admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:09:00.877977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 640/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Sat Jun 4 09:09:00 2022] POST /identity/v3/endpoints => generated 331 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:09:10.619175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=30 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:09:10.872679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:16:57.006917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f6dfa-82e6-45f0-9ced-f1dcfb3fe8fc None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:16:57.008053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f6dfa-82e6-45f0-9ced-f1dcfb3fe8fc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:16:57.008053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f6dfa-82e6-45f0-9ced-f1dcfb3fe8fc None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:16:57.009789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 667/1307] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:16:57.015334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad4a56-1591-432d-8e1b-86750351697b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:16:57.016101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad4a56-1591-432d-8e1b-86750351697b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:16:57.016101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad4a56-1591-432d-8e1b-86750351697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:16:57.120025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9bad4a56-1591-432d-8e1b-86750351697b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:16:57.121580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9bad4a56-1591-432d-8e1b-86750351697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:16:57.308215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 641/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:16:57.385447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20bd16-9bf0-4958-b636-e4e8360aaf78 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:16:57.386599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20bd16-9bf0-4958-b636-e4e8360aaf78 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:16:57.387115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20bd16-9bf0-4958-b636-e4e8360aaf78 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:16:57.390375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 668/1309] 10.222.0.46 () {54 vars in 964 bytes} [Sat Jun 4 09:16:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:16:57.398717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634001bf-62f6-4193-8cfd-9d3b50010d23 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:16:57.401648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634001bf-62f6-4193-8cfd-9d3b50010d23 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:16:57.401648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634001bf-62f6-4193-8cfd-9d3b50010d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:16:57.519222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-634001bf-62f6-4193-8cfd-9d3b50010d23 None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:16:57.521863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-634001bf-62f6-4193-8cfd-9d3b50010d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:16:57.549517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:16:57.669447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 642/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Sat Jun 4 09:16:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:16:57.686312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20bd16-9bf0-4958-b636-e4e8360aaf78 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:16:57.689639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0e20bd16-9bf0-4958-b636-e4e8360aaf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:16:57.926810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20bd16-9bf0-4958-b636-e4e8360aaf78 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:16:58.086945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:16:58.088041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:16:58.088552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:16:58.089047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:16:58.092651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:16:58.209712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:16:58.339919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:16:58.341680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3775c-42f5-4db3-bbfc-02487644babc service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:16:58.367588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 669/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:01.389786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791f6aa-1d9f-47cc-b97f-b2b603562627 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:01.390627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791f6aa-1d9f-47cc-b97f-b2b603562627 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:01.390627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791f6aa-1d9f-47cc-b97f-b2b603562627 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:01.392707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 643/1312] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:01.398314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8357b-4b76-48e4-b458-b0085a6df54d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:01.399127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8357b-4b76-48e4-b458-b0085a6df54d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:01.399127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8357b-4b76-48e4-b458-b0085a6df54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:01.485170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-53b8357b-4b76-48e4-b458-b0085a6df54d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:01.486463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-53b8357b-4b76-48e4-b458-b0085a6df54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:01.623151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 670/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:17:01.680967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298c526-a161-4792-87a2-5ae8363d99d7 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:01.681619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298c526-a161-4792-87a2-5ae8363d99d7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:01.682111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298c526-a161-4792-87a2-5ae8363d99d7 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:01.685148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 644/1314] 10.222.0.46 () {54 vars in 964 bytes} [Sat Jun 4 09:17:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:01.691654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede4a60-a821-4faf-bf5b-95effebc6fac None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:01.692279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede4a60-a821-4faf-bf5b-95effebc6fac None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:01.692738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede4a60-a821-4faf-bf5b-95effebc6fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:01.704958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:01.935638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-dede4a60-a821-4faf-bf5b-95effebc6fac None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:01.938013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dede4a60-a821-4faf-bf5b-95effebc6fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:02.105546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 671/1315] 10.222.0.46 () {58 vars in 1061 bytes} [Sat Jun 4 09:17:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:17:02.122191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298c526-a161-4792-87a2-5ae8363d99d7 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:02.125054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0298c526-a161-4792-87a2-5ae8363d99d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:02.373131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298c526-a161-4792-87a2-5ae8363d99d7 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:02.504474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:02.510707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:02.512032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:02.512544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:02.512991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:02.516676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:02.720592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:02.723351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d47e52-18ca-4fda-8b22-6a23ba7aa91a service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:02.762919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 645/1316] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:06.404129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074aeb8-24e5-4e2e-a052-d67984dc88c1 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:06.404707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074aeb8-24e5-4e2e-a052-d67984dc88c1 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:06.404707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074aeb8-24e5-4e2e-a052-d67984dc88c1 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:06.406090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 672/1317] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:06.412481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b676d5-7206-4b3f-9b58-ca060b350396 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:06.412984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b676d5-7206-4b3f-9b58-ca060b350396 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:06.413123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b676d5-7206-4b3f-9b58-ca060b350396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:06.495592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-36b676d5-7206-4b3f-9b58-ca060b350396 None None] MFA Rules not processed for user `7fa620e446a2472186b5f2e19bc28fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:06.496958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-36b676d5-7206-4b3f-9b58-ca060b350396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:06.555696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:06.613448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 646/1318] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:06.660189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074aeb8-24e5-4e2e-a052-d67984dc88c1 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:06.662634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9074aeb8-24e5-4e2e-a052-d67984dc88c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:06.809026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074aeb8-24e5-4e2e-a052-d67984dc88c1 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:06.815280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:06.914944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:06.915603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:06.915877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:06.916147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:06.918322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:07.050322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:07.051953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27d53-aac7-45a3-a6f2-21132342096e service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:07.074259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 673/1319] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:17:10.781867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f5118-c659-4092-9255-27d4f1d01c9a None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:10.782592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f5118-c659-4092-9255-27d4f1d01c9a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:10.782592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f5118-c659-4092-9255-27d4f1d01c9a None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:10.783884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 647/1320] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:17:10.790731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5cc69-f0b5-4957-b693-03dcf03858d2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:10.791552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5cc69-f0b5-4957-b693-03dcf03858d2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:10.791719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5cc69-f0b5-4957-b693-03dcf03858d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:10.840650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:10.872380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-dbe5cc69-f0b5-4957-b693-03dcf03858d2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:10.873642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dbe5cc69-f0b5-4957-b693-03dcf03858d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:10.991233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 674/1321] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:11.039241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f5118-c659-4092-9255-27d4f1d01c9a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:11.040981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-432f5118-c659-4092-9255-27d4f1d01c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:11.167789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f5118-c659-4092-9255-27d4f1d01c9a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:11.257293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:11.258074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:11.258399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:11.258737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:11.260665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:11.269786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:11.430998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:11.432600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c9c79-8ba0-41f7-ac0d-fc66a98bf1f3 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:11.457742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 648/1322] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:17:15.848844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8718a-ef0f-4a81-b8a0-0908f9155012 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:15.849724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8718a-ef0f-4a81-b8a0-0908f9155012 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:15.849881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8718a-ef0f-4a81-b8a0-0908f9155012 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:15.851820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 675/1323] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:15.859163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7023b-b7ff-4e2d-8e5c-c175f259279d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:15.859863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7023b-b7ff-4e2d-8e5c-c175f259279d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:15.860144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7023b-b7ff-4e2d-8e5c-c175f259279d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:15.957657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-78f7023b-b7ff-4e2d-8e5c-c175f259279d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:15.958916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-78f7023b-b7ff-4e2d-8e5c-c175f259279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:16.031780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:16.084009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 649/1324] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:16.149539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8718a-ef0f-4a81-b8a0-0908f9155012 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:16.151158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7fb8718a-ef0f-4a81-b8a0-0908f9155012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:16.348507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8718a-ef0f-4a81-b8a0-0908f9155012 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:16.375077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:16.498971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:16.500210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:16.500690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:16.501191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:16.504406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:16.680642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:16.682150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10951e-3961-400c-963e-254a59c36b8c service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:16.706001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 676/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:17:19.425860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a352374-d2da-485b-8743-8ab8f3cb1e91 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:19.426532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a352374-d2da-485b-8743-8ab8f3cb1e91 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:19.426532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a352374-d2da-485b-8743-8ab8f3cb1e91 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:19.428424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 650/1326] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:19.434568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4e8f3-9853-499e-93d5-e6c8636d95ec None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:19.435177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4e8f3-9853-499e-93d5-e6c8636d95ec None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:19.435177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4e8f3-9853-499e-93d5-e6c8636d95ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:19.489960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:19.514296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c6b4e8f3-9853-499e-93d5-e6c8636d95ec None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:19.515638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c6b4e8f3-9853-499e-93d5-e6c8636d95ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:19.674106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 677/1327] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:19.717510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a352374-d2da-485b-8743-8ab8f3cb1e91 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:19.719486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5a352374-d2da-485b-8743-8ab8f3cb1e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:19.865087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a352374-d2da-485b-8743-8ab8f3cb1e91 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:19.984218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:19.984818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:19.985109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:19.985372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:19.987296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:20.008272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:20.166772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:20.169689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d06a-109c-4f09-b617-fac68153b006 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:20.207564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 651/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:17:22.831121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d6ca8-3b3f-41e0-8165-62e985c156dc None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:22.831922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d6ca8-3b3f-41e0-8165-62e985c156dc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:22.831922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d6ca8-3b3f-41e0-8165-62e985c156dc None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:22.834646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 678/1329] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:22.842373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac4a7-d409-4409-b888-225a37f34c07 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:22.843139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac4a7-d409-4409-b888-225a37f34c07 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:22.843526 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac4a7-d409-4409-b888-225a37f34c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:22.966532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ec7ac4a7-d409-4409-b888-225a37f34c07 None None] MFA Rules not processed for user `7fa620e446a2472186b5f2e19bc28fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:22.968227 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ec7ac4a7-d409-4409-b888-225a37f34c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:23.058407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:23.100763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 652/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:23.156104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d6ca8-3b3f-41e0-8165-62e985c156dc None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:23.157875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2e3d6ca8-3b3f-41e0-8165-62e985c156dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:23.291654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d6ca8-3b3f-41e0-8165-62e985c156dc None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:23.318681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:23.388266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:23.389055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:23.389404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:23.389839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:23.391959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:23.546388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:23.548092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f8487-29f7-4ad6-b311-68b48fdcbcbe service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:23.571405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 679/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:26.710069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79786ca9-3d97-4940-ae9f-2762ab22876e None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:26.710628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79786ca9-3d97-4940-ae9f-2762ab22876e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:26.710628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79786ca9-3d97-4940-ae9f-2762ab22876e None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:26.712010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 653/1332] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:26.718112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad0b34-a14d-4997-9945-ae0779c0a391 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:26.733292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad0b34-a14d-4997-9945-ae0779c0a391 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:26.733292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad0b34-a14d-4997-9945-ae0779c0a391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:26.789112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:26.792340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-12ad0b34-a14d-4997-9945-ae0779c0a391 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:26.798850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-12ad0b34-a14d-4997-9945-ae0779c0a391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:26.912262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 680/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:26.967762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79786ca9-3d97-4940-ae9f-2762ab22876e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:26.969351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-79786ca9-3d97-4940-ae9f-2762ab22876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:27.115646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79786ca9-3d97-4940-ae9f-2762ab22876e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:27.243510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:27.243950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:27.244241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:27.244570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:27.246900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:27.436159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:27.569789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:27.571292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f38e8a-6f43-463c-92fe-645b89b90985 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:27.605982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 654/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:30.366850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e76062-a1a1-4a32-ac83-615133dd735d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:30.367526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e76062-a1a1-4a32-ac83-615133dd735d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:30.367526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e76062-a1a1-4a32-ac83-615133dd735d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:30.368823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 681/1335] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:30.374773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ea94c-041c-4ca5-8451-cf0716559857 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:30.375359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ea94c-041c-4ca5-8451-cf0716559857 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:30.375359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ea94c-041c-4ca5-8451-cf0716559857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:30.427243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-534ea94c-041c-4ca5-8451-cf0716559857 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:30.429204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-534ea94c-041c-4ca5-8451-cf0716559857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:30.438042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 655/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:30.479893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e76062-a1a1-4a32-ac83-615133dd735d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:30.482516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c2e76062-a1a1-4a32-ac83-615133dd735d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:30.521687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e76062-a1a1-4a32-ac83-615133dd735d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:30.525204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142aba3-8563-4823-86ad-d8c954062f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:30.525854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142aba3-8563-4823-86ad-d8c954062f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:30.526136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142aba3-8563-4823-86ad-d8c954062f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:30.526411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142aba3-8563-4823-86ad-d8c954062f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:30.536638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142aba3-8563-4823-86ad-d8c954062f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:30.538624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142aba3-8563-4823-86ad-d8c954062f30 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:30.549087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 682/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:33.653535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb50261-b448-444c-8715-d400987dcd4f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:33.654236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb50261-b448-444c-8715-d400987dcd4f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:33.654236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb50261-b448-444c-8715-d400987dcd4f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:33.655503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 656/1338] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:33.661325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10ac18-e1d3-4941-a3c9-6f9199965278 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:33.661851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10ac18-e1d3-4941-a3c9-6f9199965278 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:33.661970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10ac18-e1d3-4941-a3c9-6f9199965278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:33.692582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ca10ac18-e1d3-4941-a3c9-6f9199965278 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:33.693851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ca10ac18-e1d3-4941-a3c9-6f9199965278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:33.788636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 683/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:33.825575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb50261-b448-444c-8715-d400987dcd4f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:33.827485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8bb50261-b448-444c-8715-d400987dcd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:33.841879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb50261-b448-444c-8715-d400987dcd4f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:33.845581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6e677-cdc6-4108-9232-4a32a2554b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:33.846248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6e677-cdc6-4108-9232-4a32a2554b21 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:33.846529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6e677-cdc6-4108-9232-4a32a2554b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:33.846813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6e677-cdc6-4108-9232-4a32a2554b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:33.857670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6e677-cdc6-4108-9232-4a32a2554b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:33.859136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6e677-cdc6-4108-9232-4a32a2554b21 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:33.870141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 657/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:38.743482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5081b2-78af-4c65-89a3-c5f6666548a2 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:38.744359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5081b2-78af-4c65-89a3-c5f6666548a2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:38.744359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5081b2-78af-4c65-89a3-c5f6666548a2 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:38.745696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 684/1341] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:38.753575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2cf66-35d5-41fc-adca-89b55019b068 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:38.754192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2cf66-35d5-41fc-adca-89b55019b068 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:38.754819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2cf66-35d5-41fc-adca-89b55019b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:38.777184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-33d2cf66-35d5-41fc-adca-89b55019b068 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:38.778528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-33d2cf66-35d5-41fc-adca-89b55019b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:38.788665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 658/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:38.831400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5081b2-78af-4c65-89a3-c5f6666548a2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:38.843476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5081b2-78af-4c65-89a3-c5f6666548a2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:38.848255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae3a7b-4ec9-4dc2-bc5f-c425744c28ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:38.849038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3a7b-4ec9-4dc2-bc5f-c425744c28ac service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:38.849387 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3a7b-4ec9-4dc2-bc5f-c425744c28ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:38.849750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3a7b-4ec9-4dc2-bc5f-c425744c28ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:38.856857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:17:38.860989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3a7b-4ec9-4dc2-bc5f-c425744c28ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:38.862592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3a7b-4ec9-4dc2-bc5f-c425744c28ac service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:38.874107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 685/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:43.135286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5bfb7-2b1a-4c40-98fd-68c7e0b9e2ed None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:43.135871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5bfb7-2b1a-4c40-98fd-68c7e0b9e2ed None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:43.135871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5bfb7-2b1a-4c40-98fd-68c7e0b9e2ed None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:43.137133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 659/1344] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:17:43.144735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7f41-4243-4aa8-a488-e60c45f2d031 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:43.145281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7f41-4243-4aa8-a488-e60c45f2d031 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:43.145281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7f41-4243-4aa8-a488-e60c45f2d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:43.165404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-05cf7f41-4243-4aa8-a488-e60c45f2d031 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:43.166630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-05cf7f41-4243-4aa8-a488-e60c45f2d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:43.175019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 686/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:43.213763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5bfb7-2b1a-4c40-98fd-68c7e0b9e2ed None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:43.227455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5bfb7-2b1a-4c40-98fd-68c7e0b9e2ed None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:43.237741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10945a-c367-4f1c-9215-cda2d5461aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:43.238437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10945a-c367-4f1c-9215-cda2d5461aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:43.238807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10945a-c367-4f1c-9215-cda2d5461aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:43.239085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10945a-c367-4f1c-9215-cda2d5461aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:43.249955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10945a-c367-4f1c-9215-cda2d5461aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:43.251437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10945a-c367-4f1c-9215-cda2d5461aa5 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:43.262096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 660/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:48.242287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ce0c0-0c80-4f7f-b4ab-650315e3a091 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:48.243005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ce0c0-0c80-4f7f-b4ab-650315e3a091 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:48.243005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ce0c0-0c80-4f7f-b4ab-650315e3a091 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:48.244140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 687/1347] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:48.250963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f198ac2-2187-4865-b3f9-0e9c5f7c9037 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:48.251693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f198ac2-2187-4865-b3f9-0e9c5f7c9037 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:48.251916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f198ac2-2187-4865-b3f9-0e9c5f7c9037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:48.278443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-1f198ac2-2187-4865-b3f9-0e9c5f7c9037 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:48.279812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1f198ac2-2187-4865-b3f9-0e9c5f7c9037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:48.290192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 661/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:48.332040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ce0c0-0c80-4f7f-b4ab-650315e3a091 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:17:48.345082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ce0c0-0c80-4f7f-b4ab-650315e3a091 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:17:48.348920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef429d-9eae-463f-99fd-e53382693624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:17:48.349585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef429d-9eae-463f-99fd-e53382693624 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:48.349868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef429d-9eae-463f-99fd-e53382693624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:48.350146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef429d-9eae-463f-99fd-e53382693624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:48.360577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef429d-9eae-463f-99fd-e53382693624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:17:48.362039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef429d-9eae-463f-99fd-e53382693624 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:17:48.373249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 688/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:17:55.190158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6dcbe-28b4-469f-afcb-1b31c4dec408 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:55.190688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6dcbe-28b4-469f-afcb-1b31c4dec408 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:55.190769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6dcbe-28b4-469f-afcb-1b31c4dec408 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:55.192737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 662/1350] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:17:55.218473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045b9d7-0371-42fb-82e6-a2c012f8591b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:17:55.219026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045b9d7-0371-42fb-82e6-a2c012f8591b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:17:55.219632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045b9d7-0371-42fb-82e6-a2c012f8591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:17:55.435820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2045b9d7-0371-42fb-82e6-a2c012f8591b None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:17:55.437057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2045b9d7-0371-42fb-82e6-a2c012f8591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:17:55.530702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 689/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:17:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:17:58.271578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=74 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:18:00.678248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e203d00-e3b6-4201-b9fd-9389d4ddf860 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:00.678248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e203d00-e3b6-4201-b9fd-9389d4ddf860 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:00.678248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e203d00-e3b6-4201-b9fd-9389d4ddf860 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:00.680178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 663/1352] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:00.687457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fbacf-551a-439b-8e83-cfe2cf386b97 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:00.689303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fbacf-551a-439b-8e83-cfe2cf386b97 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:00.690114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fbacf-551a-439b-8e83-cfe2cf386b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:00.732473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d93fbacf-551a-439b-8e83-cfe2cf386b97 None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:00.735861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d93fbacf-551a-439b-8e83-cfe2cf386b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:00.756274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 690/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:18:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:02.022100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c580f4-0b9d-4797-895e-e4e56e4272c9 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:02.023733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c580f4-0b9d-4797-895e-e4e56e4272c9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:02.023733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c580f4-0b9d-4797-895e-e4e56e4272c9 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:02.025014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 664/1354] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:18:02.031652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25fe9e-017d-4e1b-9437-66226e7e26ac None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:02.032458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25fe9e-017d-4e1b-9437-66226e7e26ac None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:02.032954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25fe9e-017d-4e1b-9437-66226e7e26ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:02.064188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3a25fe9e-017d-4e1b-9437-66226e7e26ac None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:02.065623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3a25fe9e-017d-4e1b-9437-66226e7e26ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:02.077378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 691/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:06.780860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7848b-5281-41c1-af14-71eca4917a4d None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:06.781455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7848b-5281-41c1-af14-71eca4917a4d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:06.781455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7848b-5281-41c1-af14-71eca4917a4d None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:06.787384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 665/1356] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:06.789158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc72aee-441d-4b35-80c1-5cf3dffc3a18 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:06.789913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc72aee-441d-4b35-80c1-5cf3dffc3a18 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:06.789913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc72aee-441d-4b35-80c1-5cf3dffc3a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:06.821159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3dc72aee-441d-4b35-80c1-5cf3dffc3a18 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:06.822738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3dc72aee-441d-4b35-80c1-5cf3dffc3a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:06.832718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 692/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:06.919470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c8f80-246b-4013-8eaf-329653585eb0 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:06.919748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c8f80-246b-4013-8eaf-329653585eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:06.920017 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c8f80-246b-4013-8eaf-329653585eb0 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:06.922043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 666/1358] 10.222.0.46 () {54 vars in 961 bytes} [Sat Jun 4 09:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:06.931076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ea993-e97d-4e84-8d5b-8c10b1d807a6 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:06.931896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ea993-e97d-4e84-8d5b-8c10b1d807a6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:06.932405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ea993-e97d-4e84-8d5b-8c10b1d807a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:06.993949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6e9ea993-e97d-4e84-8d5b-8c10b1d807a6 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:06.996406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6e9ea993-e97d-4e84-8d5b-8c10b1d807a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:07.063650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:18:07.179997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 693/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Sat Jun 4 09:18:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:07.187136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c8f80-246b-4013-8eaf-329653585eb0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:07.216033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c8f80-246b-4013-8eaf-329653585eb0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:07.227195 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d25979-956b-44c9-942d-ba86c2f03a6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:07.228332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d25979-956b-44c9-942d-ba86c2f03a6b service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:07.229042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d25979-956b-44c9-942d-ba86c2f03a6b service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:07.229828 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d25979-956b-44c9-942d-ba86c2f03a6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:07.247867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d25979-956b-44c9-942d-ba86c2f03a6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:07.251056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d25979-956b-44c9-942d-ba86c2f03a6b service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:07.276214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 667/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:10.377098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367c22-96b9-42b1-aa11-9838ce6410d9 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:10.377927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367c22-96b9-42b1-aa11-9838ce6410d9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:10.377927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367c22-96b9-42b1-aa11-9838ce6410d9 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:10.380488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 694/1361] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:18:10.385762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6bca-d9a7-4a82-a9f1-f809ebc86b04 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:10.386540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6bca-d9a7-4a82-a9f1-f809ebc86b04 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:10.386540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6bca-d9a7-4a82-a9f1-f809ebc86b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:10.407935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-da6f6bca-d9a7-4a82-a9f1-f809ebc86b04 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:10.409229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-da6f6bca-d9a7-4a82-a9f1-f809ebc86b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:10.418478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 668/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:10.465519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367c22-96b9-42b1-aa11-9838ce6410d9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:10.482499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367c22-96b9-42b1-aa11-9838ce6410d9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:10.488860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87401b5-d5bd-424a-8990-fc74f7bbe5af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:10.489611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87401b5-d5bd-424a-8990-fc74f7bbe5af service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:10.489964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87401b5-d5bd-424a-8990-fc74f7bbe5af service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:10.490314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87401b5-d5bd-424a-8990-fc74f7bbe5af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:10.501163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87401b5-d5bd-424a-8990-fc74f7bbe5af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:10.502874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87401b5-d5bd-424a-8990-fc74f7bbe5af service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:10.515343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 695/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:13.849641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff84cd-a711-498b-bb40-13c93f3f206e None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:13.850250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff84cd-a711-498b-bb40-13c93f3f206e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:13.850250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff84cd-a711-498b-bb40-13c93f3f206e None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:13.852186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 669/1364] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:13.858940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73027-96cb-49f7-a164-2d82b8d16584 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:13.859504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73027-96cb-49f7-a164-2d82b8d16584 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:13.860279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73027-96cb-49f7-a164-2d82b8d16584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:13.901867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-47e73027-96cb-49f7-a164-2d82b8d16584 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:13.903789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-47e73027-96cb-49f7-a164-2d82b8d16584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:13.916994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 696/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:13.985272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab502b1-cdde-46eb-ad40-5263ae0aca12 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:13.985729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab502b1-cdde-46eb-ad40-5263ae0aca12 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:13.986095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab502b1-cdde-46eb-ad40-5263ae0aca12 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:13.987703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 670/1366] 10.222.0.46 () {54 vars in 961 bytes} [Sat Jun 4 09:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:13.997136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7f78-b96d-4660-b50b-77f2e05cdf97 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:13.997749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7f78-b96d-4660-b50b-77f2e05cdf97 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:13.998289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7f78-b96d-4660-b50b-77f2e05cdf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:14.025882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-745c7f78-b96d-4660-b50b-77f2e05cdf97 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:14.027506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-745c7f78-b96d-4660-b50b-77f2e05cdf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:14.043894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 697/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Sat Jun 4 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:14.055908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab502b1-cdde-46eb-ad40-5263ae0aca12 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:14.079661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab502b1-cdde-46eb-ad40-5263ae0aca12 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:14.090875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eef9a-5b42-418a-9821-bba506eb29a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:14.091979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eef9a-5b42-418a-9821-bba506eb29a8 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:14.092562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eef9a-5b42-418a-9821-bba506eb29a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:14.093946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eef9a-5b42-418a-9821-bba506eb29a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:14.111412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eef9a-5b42-418a-9821-bba506eb29a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:14.114585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eef9a-5b42-418a-9821-bba506eb29a8 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:14.136506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 671/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:18:16.646363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831bf6-06b3-4d24-846f-2643d43098b4 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:16.647281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831bf6-06b3-4d24-846f-2643d43098b4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:16.647281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831bf6-06b3-4d24-846f-2643d43098b4 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:16.649041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 698/1369] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:16.654955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a4519-8559-4eb0-aaca-acea4d0af33f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:16.655726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a4519-8559-4eb0-aaca-acea4d0af33f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:16.655726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a4519-8559-4eb0-aaca-acea4d0af33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:16.675430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-652a4519-8559-4eb0-aaca-acea4d0af33f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:16.676712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-652a4519-8559-4eb0-aaca-acea4d0af33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:16.691766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 672/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:16.972346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335308a-7759-4761-b900-581e26c86818 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:16.972672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335308a-7759-4761-b900-581e26c86818 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:16.972897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335308a-7759-4761-b900-581e26c86818 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:16.974511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 699/1371] 10.222.0.46 () {54 vars in 963 bytes} [Sat Jun 4 09:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:18:16.981830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc61d6-59f7-452b-b8be-5848925b1157 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:16.982669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc61d6-59f7-452b-b8be-5848925b1157 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:16.983228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc61d6-59f7-452b-b8be-5848925b1157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:17.082827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-6cfc61d6-59f7-452b-b8be-5848925b1157 None None] MFA Rules not processed for user `e53e659f7faf403b9f33f393d166dda3`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:17.085208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6cfc61d6-59f7-452b-b8be-5848925b1157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:17.174043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 673/1372] 10.222.0.46 () {58 vars in 1060 bytes} [Sat Jun 4 09:18:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:17.182255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335308a-7759-4761-b900-581e26c86818 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:17.194941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335308a-7759-4761-b900-581e26c86818 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:17.198857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d0018-153b-4a89-a0b9-0a09f7a230e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:17.199510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d0018-153b-4a89-a0b9-0a09f7a230e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:17.199810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d0018-153b-4a89-a0b9-0a09f7a230e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:17.200309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d0018-153b-4a89-a0b9-0a09f7a230e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:17.212070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d0018-153b-4a89-a0b9-0a09f7a230e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:17.213637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d0018-153b-4a89-a0b9-0a09f7a230e6 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:17.225879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 700/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:18:17.950591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc300c3f-7a00-4c3c-af1c-f9d83d260700 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:17.951420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc300c3f-7a00-4c3c-af1c-f9d83d260700 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:17.951420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc300c3f-7a00-4c3c-af1c-f9d83d260700 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:17.956346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 674/1374] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:17.962299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe964db-a28f-4371-a0e4-e25b0ce42d56 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:17.963115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe964db-a28f-4371-a0e4-e25b0ce42d56 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:17.963115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe964db-a28f-4371-a0e4-e25b0ce42d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:17.983750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bfe964db-a28f-4371-a0e4-e25b0ce42d56 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:17.985115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bfe964db-a28f-4371-a0e4-e25b0ce42d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:17.993934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 701/1375] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:18.045158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc300c3f-7a00-4c3c-af1c-f9d83d260700 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:18.061645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc300c3f-7a00-4c3c-af1c-f9d83d260700 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:18.068042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0c1e-c398-43d2-b7db-c894ed95462f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:18.068780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0c1e-c398-43d2-b7db-c894ed95462f service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:18.069117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0c1e-c398-43d2-b7db-c894ed95462f service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:18.069377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0c1e-c398-43d2-b7db-c894ed95462f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:18.080135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a0c1e-c398-43d2-b7db-c894ed95462f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:18.081874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a0c1e-c398-43d2-b7db-c894ed95462f service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:18.095065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 675/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:21.081304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1475b-8491-41cd-8111-045aaea63f89 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:21.081947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1475b-8491-41cd-8111-045aaea63f89 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:21.081947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1475b-8491-41cd-8111-045aaea63f89 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:21.083405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 702/1377] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:21.090792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2633b-6276-421e-aa54-09c6f86c7e8c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:21.091649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2633b-6276-421e-aa54-09c6f86c7e8c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:21.091809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2633b-6276-421e-aa54-09c6f86c7e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:21.118299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b0c2633b-6276-421e-aa54-09c6f86c7e8c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:21.119780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b0c2633b-6276-421e-aa54-09c6f86c7e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:21.135527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 676/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:21.211465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1475b-8491-41cd-8111-045aaea63f89 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:21.230388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1475b-8491-41cd-8111-045aaea63f89 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:21.237770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7242e1-331f-4741-83ef-c0722ee748ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:21.238543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7242e1-331f-4741-83ef-c0722ee748ad service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:21.238873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7242e1-331f-4741-83ef-c0722ee748ad service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:21.239183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7242e1-331f-4741-83ef-c0722ee748ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:21.249895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7242e1-331f-4741-83ef-c0722ee748ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:21.251947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7242e1-331f-4741-83ef-c0722ee748ad service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:21.266063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 703/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:24.797920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b7923-a593-4ef5-91dc-421db0f69941 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:24.799633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b7923-a593-4ef5-91dc-421db0f69941 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:24.800375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b7923-a593-4ef5-91dc-421db0f69941 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:24.803507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 677/1380] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:24.813123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7d1a-9cb6-4ffa-8cd2-648a8bbdd42d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:24.813935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7d1a-9cb6-4ffa-8cd2-648a8bbdd42d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:24.813935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e7d1a-9cb6-4ffa-8cd2-648a8bbdd42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:24.834123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a45e7d1a-9cb6-4ffa-8cd2-648a8bbdd42d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:24.835348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a45e7d1a-9cb6-4ffa-8cd2-648a8bbdd42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:24.844098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 704/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:24.893071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b7923-a593-4ef5-91dc-421db0f69941 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:24.909140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b7923-a593-4ef5-91dc-421db0f69941 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:24.915415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fbfb6-8882-4b8b-a092-be89649c0b48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:24.916071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fbfb6-8882-4b8b-a092-be89649c0b48 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:24.916372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fbfb6-8882-4b8b-a092-be89649c0b48 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:24.916843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fbfb6-8882-4b8b-a092-be89649c0b48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:24.926748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fbfb6-8882-4b8b-a092-be89649c0b48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:24.928655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fbfb6-8882-4b8b-a092-be89649c0b48 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:24.941028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 678/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:18:28.578300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd3c4-932f-4957-83b7-5aa28b536786 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:28.578300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd3c4-932f-4957-83b7-5aa28b536786 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:28.578300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd3c4-932f-4957-83b7-5aa28b536786 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:28.580795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 705/1383] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:28.589852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f7817-8994-4dff-a31d-a5795a8c5e97 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:28.590601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f7817-8994-4dff-a31d-a5795a8c5e97 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:28.591620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f7817-8994-4dff-a31d-a5795a8c5e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:28.605438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=72 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:18:28.631920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-386f7817-8994-4dff-a31d-a5795a8c5e97 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:28.635987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-386f7817-8994-4dff-a31d-a5795a8c5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:28.663554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 679/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:28.751520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd3c4-932f-4957-83b7-5aa28b536786 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:28.777222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd3c4-932f-4957-83b7-5aa28b536786 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:28.791565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e795c7-5564-4b7d-a4b9-34baef31afe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:28.792703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e795c7-5564-4b7d-a4b9-34baef31afe3 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:28.793197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e795c7-5564-4b7d-a4b9-34baef31afe3 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:28.795472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e795c7-5564-4b7d-a4b9-34baef31afe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:28.816198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e795c7-5564-4b7d-a4b9-34baef31afe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:28.819951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e795c7-5564-4b7d-a4b9-34baef31afe3 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:28.838450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 706/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:31.728496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d042f-5382-4c31-b0e1-72608bcdb1c5 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:31.729346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d042f-5382-4c31-b0e1-72608bcdb1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:31.729978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d042f-5382-4c31-b0e1-72608bcdb1c5 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:31.733771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 680/1386] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:31.740673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431033bf-73d6-439b-ae57-0e2820637914 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:31.742740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431033bf-73d6-439b-ae57-0e2820637914 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:31.743239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431033bf-73d6-439b-ae57-0e2820637914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:31.774675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-431033bf-73d6-439b-ae57-0e2820637914 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:31.776411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-431033bf-73d6-439b-ae57-0e2820637914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:31.787659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 707/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:31.870976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d042f-5382-4c31-b0e1-72608bcdb1c5 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:31.891984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d042f-5382-4c31-b0e1-72608bcdb1c5 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:31.902006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabef4d-0277-4caf-bdf0-7e9bd86f12e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:31.903590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabef4d-0277-4caf-bdf0-7e9bd86f12e2 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:31.904327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabef4d-0277-4caf-bdf0-7e9bd86f12e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:31.904989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabef4d-0277-4caf-bdf0-7e9bd86f12e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:31.928739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabef4d-0277-4caf-bdf0-7e9bd86f12e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:31.933875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabef4d-0277-4caf-bdf0-7e9bd86f12e2 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:31.961174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 681/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:18:35.144236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d08dab-ecb0-4ba7-9853-e472b3269112 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:35.145184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d08dab-ecb0-4ba7-9853-e472b3269112 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:35.145184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d08dab-ecb0-4ba7-9853-e472b3269112 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:35.146335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 708/1389] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:18:35.156955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd1dc-313d-4cc4-966f-96a3f1877b24 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:35.157581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd1dc-313d-4cc4-966f-96a3f1877b24 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:35.158126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd1dc-313d-4cc4-966f-96a3f1877b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:35.184263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-773dd1dc-313d-4cc4-966f-96a3f1877b24 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:35.185687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-773dd1dc-313d-4cc4-966f-96a3f1877b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:35.198276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 682/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:35.261869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d08dab-ecb0-4ba7-9853-e472b3269112 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:35.288696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d08dab-ecb0-4ba7-9853-e472b3269112 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:35.299843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da36e-2eba-4299-a6a6-7a98904117ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:35.301033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da36e-2eba-4299-a6a6-7a98904117ac service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:35.301545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da36e-2eba-4299-a6a6-7a98904117ac service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:35.302060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da36e-2eba-4299-a6a6-7a98904117ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:35.319185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25da36e-2eba-4299-a6a6-7a98904117ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:35.323223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25da36e-2eba-4299-a6a6-7a98904117ac service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:35.342634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 709/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:36.949002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b349-c959-4e78-a6c3-88f23dacd6b2 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:36.949823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b349-c959-4e78-a6c3-88f23dacd6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:36.949988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b349-c959-4e78-a6c3-88f23dacd6b2 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:36.952736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 683/1392] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:36.962921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825375-130f-441d-acd2-f852d1b0319b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:36.963487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825375-130f-441d-acd2-f852d1b0319b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:36.964012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825375-130f-441d-acd2-f852d1b0319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:36.994567 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b7825375-130f-441d-acd2-f852d1b0319b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:36.995809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b7825375-130f-441d-acd2-f852d1b0319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:37.006023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 710/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:37.086520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca9194-fae9-4444-a0e5-4cd8d9d8d1f4 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:37.087120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca9194-fae9-4444-a0e5-4cd8d9d8d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:37.087632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca9194-fae9-4444-a0e5-4cd8d9d8d1f4 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:37.090241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 684/1394] 10.222.0.46 () {54 vars in 962 bytes} [Sat Jun 4 09:18:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:37.098641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b36f8-f549-4e7d-acea-775ec264aa60 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:37.099001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b36f8-f549-4e7d-acea-775ec264aa60 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:37.099340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b36f8-f549-4e7d-acea-775ec264aa60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:37.137299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=79 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:18:37.168118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6d4b36f8-f549-4e7d-acea-775ec264aa60 None None] MFA Rules not processed for user `8b3a9e1e2e3d438f92286a4158c8d4cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:37.170111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6d4b36f8-f549-4e7d-acea-775ec264aa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:37.275895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 711/1395] 10.222.0.46 () {58 vars in 1059 bytes} [Sat Jun 4 09:18:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:37.287070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca9194-fae9-4444-a0e5-4cd8d9d8d1f4 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:37.306478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca9194-fae9-4444-a0e5-4cd8d9d8d1f4 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:37.313005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4afbc1-a2ff-42fa-b1b7-1daf709da330 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:37.314050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4afbc1-a2ff-42fa-b1b7-1daf709da330 service glance] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:37.314512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4afbc1-a2ff-42fa-b1b7-1daf709da330 service glance] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:37.314966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4afbc1-a2ff-42fa-b1b7-1daf709da330 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:37.331733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4afbc1-a2ff-42fa-b1b7-1daf709da330 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:37.334495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4afbc1-a2ff-42fa-b1b7-1daf709da330 service glance] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:37.354359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 685/1396] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:18:38.094734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a279928-e4e5-4223-b5b9-d974cf604b11 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.097913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a279928-e4e5-4223-b5b9-d974cf604b11 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.097913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a279928-e4e5-4223-b5b9-d974cf604b11 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.097913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 712/1397] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:38.106387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de41e-2cd8-4903-8b59-a95883f88215 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.107361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de41e-2cd8-4903-8b59-a95883f88215 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.107361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de41e-2cd8-4903-8b59-a95883f88215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.130804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-386de41e-2cd8-4903-8b59-a95883f88215 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:38.132370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-386de41e-2cd8-4903-8b59-a95883f88215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:38.142744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 686/1398] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:38.197890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a279928-e4e5-4223-b5b9-d974cf604b11 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:38.214171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a279928-e4e5-4223-b5b9-d974cf604b11 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:38.221119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315e035-5a18-462f-b9f7-b0d9e995ca6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:38.221771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315e035-5a18-462f-b9f7-b0d9e995ca6a service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.222057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315e035-5a18-462f-b9f7-b0d9e995ca6a service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.222368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315e035-5a18-462f-b9f7-b0d9e995ca6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.235184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315e035-5a18-462f-b9f7-b0d9e995ca6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:38.238278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315e035-5a18-462f-b9f7-b0d9e995ca6a service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:38.251171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 713/1399] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:38.336829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8a04-0192-4d49-8360-6c5e9783bc04 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.337279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8a04-0192-4d49-8360-6c5e9783bc04 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.337745 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8a04-0192-4d49-8360-6c5e9783bc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.411468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e5fc8a04-0192-4d49-8360-6c5e9783bc04 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:38.412994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e5fc8a04-0192-4d49-8360-6c5e9783bc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:38.495059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 687/1400] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:18:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:38.567745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24fdb-08c0-4540-b398-95998424587a None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.568055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24fdb-08c0-4540-b398-95998424587a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.568409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24fdb-08c0-4540-b398-95998424587a None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.570433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 714/1401] 10.222.0.46 () {54 vars in 942 bytes} [Sat Jun 4 09:18:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:38.582491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313c0e6-051f-43d6-94bd-17da80e08685 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.582987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313c0e6-051f-43d6-94bd-17da80e08685 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.583429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313c0e6-051f-43d6-94bd-17da80e08685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.652506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d313c0e6-051f-43d6-94bd-17da80e08685 None None] MFA Rules not processed for user `85a16b32249c48eb89cf0d7cd55e5de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:38.654021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d313c0e6-051f-43d6-94bd-17da80e08685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:38.727572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:18:38.762878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 688/1402] 10.222.0.46 () {58 vars in 1039 bytes} [Sat Jun 4 09:18:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:38.778265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f24fdb-08c0-4540-b398-95998424587a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:38.805633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f24fdb-08c0-4540-b398-95998424587a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:38.812864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d468dcc-4152-4891-b8d9-0d27b5300dbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:38.813889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468dcc-4152-4891-b8d9-0d27b5300dbc service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:38.814383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468dcc-4152-4891-b8d9-0d27b5300dbc service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:38.814839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468dcc-4152-4891-b8d9-0d27b5300dbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:38.937315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d468dcc-4152-4891-b8d9-0d27b5300dbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:38.937315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d468dcc-4152-4891-b8d9-0d27b5300dbc service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:38.937315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 715/1403] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:18:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:18:40.934736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc028423-e37c-45fc-a990-7a3daf489921 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:40.935313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc028423-e37c-45fc-a990-7a3daf489921 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:40.935466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc028423-e37c-45fc-a990-7a3daf489921 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:40.937145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 689/1404] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:18:40.943871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502b6c-0b25-454c-89d1-a3c53bfd6972 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:40.944656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502b6c-0b25-454c-89d1-a3c53bfd6972 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:40.944656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2502b6c-0b25-454c-89d1-a3c53bfd6972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:40.969049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e2502b6c-0b25-454c-89d1-a3c53bfd6972 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:40.970439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e2502b6c-0b25-454c-89d1-a3c53bfd6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:40.986552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 716/1405] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:18:41.037602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc028423-e37c-45fc-a990-7a3daf489921 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:41.053388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc028423-e37c-45fc-a990-7a3daf489921 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:41.061132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02749266-7c9e-4006-8979-32242c4e90a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:41.061944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749266-7c9e-4006-8979-32242c4e90a8 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:41.062243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749266-7c9e-4006-8979-32242c4e90a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:41.062685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749266-7c9e-4006-8979-32242c4e90a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:41.073405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02749266-7c9e-4006-8979-32242c4e90a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:41.075554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02749266-7c9e-4006-8979-32242c4e90a8 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:41.087655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 690/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:44.168302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffea1ee-5fc4-4e61-9110-696a2428a118 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:44.168853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffea1ee-5fc4-4e61-9110-696a2428a118 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:44.169191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffea1ee-5fc4-4e61-9110-696a2428a118 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:44.171229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 717/1407] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:18:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:18:44.179700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05277c4f-4729-4803-9635-eb061c86cb6b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:44.180427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05277c4f-4729-4803-9635-eb061c86cb6b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:44.180733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05277c4f-4729-4803-9635-eb061c86cb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:44.219918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-05277c4f-4729-4803-9635-eb061c86cb6b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:18:44.222066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-05277c4f-4729-4803-9635-eb061c86cb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:18:44.235081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 691/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:18:44.301550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffea1ee-5fc4-4e61-9110-696a2428a118 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:18:44.320202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffea1ee-5fc4-4e61-9110-696a2428a118 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:18:44.332575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964106f8-f86f-4230-ad0c-a7c2f4dfd65e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:18:44.333843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964106f8-f86f-4230-ad0c-a7c2f4dfd65e service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:18:44.334309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964106f8-f86f-4230-ad0c-a7c2f4dfd65e service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:18:44.334594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964106f8-f86f-4230-ad0c-a7c2f4dfd65e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:18:44.346739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964106f8-f86f-4230-ad0c-a7c2f4dfd65e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:18:44.350961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964106f8-f86f-4230-ad0c-a7c2f4dfd65e service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:18:44.362576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 718/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:18:54.210524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=13 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:18:54.357869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=50 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:19:05.173642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7c4d-8169-4b24-a6e9-647eec0a9be8 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:05.174440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7c4d-8169-4b24-a6e9-647eec0a9be8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:05.174440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7c4d-8169-4b24-a6e9-647eec0a9be8 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:05.175649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 692/1410] 10.222.0.46 () {54 vars in 918 bytes} [Sat Jun 4 09:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:19:05.182213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fee5-0b3f-4635-bd2d-7c4a21537ebe None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:05.182807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fee5-0b3f-4635-bd2d-7c4a21537ebe None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:05.182807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fee5-0b3f-4635-bd2d-7c4a21537ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:05.203724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3388fee5-0b3f-4635-bd2d-7c4a21537ebe None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:05.205023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3388fee5-0b3f-4635-bd2d-7c4a21537ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:05.214647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 719/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Sat Jun 4 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:05.235455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bafeb5-aab4-4b43-a615-a37e0b1c6527 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:05.235812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bafeb5-aab4-4b43-a615-a37e0b1c6527 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:05.236203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bafeb5-aab4-4b43-a615-a37e0b1c6527 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:05.237737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 693/1412] 10.222.0.46 () {54 vars in 918 bytes} [Sat Jun 4 09:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:05.243525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b53cec-f7a0-4809-aa6b-885f94d61470 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:05.243812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b53cec-f7a0-4809-aa6b-885f94d61470 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:05.244090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b53cec-f7a0-4809-aa6b-885f94d61470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:05.263841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-03b53cec-f7a0-4809-aa6b-885f94d61470 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:05.265363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-03b53cec-f7a0-4809-aa6b-885f94d61470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:05.276770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 720/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Sat Jun 4 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:08.240710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322d55-c988-4174-bd2f-0ff0e64dd0e3 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:08.241336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322d55-c988-4174-bd2f-0ff0e64dd0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:08.241336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322d55-c988-4174-bd2f-0ff0e64dd0e3 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:08.243059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 694/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:08.250138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad977c-2bda-49ba-b897-fed287654d6f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:08.250986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad977c-2bda-49ba-b897-fed287654d6f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:08.251169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad977c-2bda-49ba-b897-fed287654d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:08.272428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-22ad977c-2bda-49ba-b897-fed287654d6f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:08.274042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-22ad977c-2bda-49ba-b897-fed287654d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:08.284729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 721/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:08.332661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4322d55-c988-4174-bd2f-0ff0e64dd0e3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:08.349853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4322d55-c988-4174-bd2f-0ff0e64dd0e3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:08.357102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102034e3-dcf8-4649-b812-bbbb75acefd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:08.357983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102034e3-dcf8-4649-b812-bbbb75acefd6 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:08.358376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102034e3-dcf8-4649-b812-bbbb75acefd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:08.358754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102034e3-dcf8-4649-b812-bbbb75acefd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:08.369112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102034e3-dcf8-4649-b812-bbbb75acefd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:08.370811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102034e3-dcf8-4649-b812-bbbb75acefd6 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:08.382643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 695/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:09.668777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c9ff-3ec8-4220-9fca-3348b3ec127d None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:09.669420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c9ff-3ec8-4220-9fca-3348b3ec127d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:09.669420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c9ff-3ec8-4220-9fca-3348b3ec127d None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:09.671134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 722/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:09.677646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a9b8-7d3a-4892-b6c3-f49b1260f098 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:09.678438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a9b8-7d3a-4892-b6c3-f49b1260f098 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:09.678438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a9b8-7d3a-4892-b6c3-f49b1260f098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:09.702365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-6b70a9b8-7d3a-4892-b6c3-f49b1260f098 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:09.703856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6b70a9b8-7d3a-4892-b6c3-f49b1260f098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:09.714115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 696/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:09.890181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c9ff-3ec8-4220-9fca-3348b3ec127d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:09.916073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c9ff-3ec8-4220-9fca-3348b3ec127d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:09.926038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd185e3a-629b-4063-a3ae-764dedb0b8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:09.927116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185e3a-629b-4063-a3ae-764dedb0b8ff service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:09.927631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185e3a-629b-4063-a3ae-764dedb0b8ff service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:09.928121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185e3a-629b-4063-a3ae-764dedb0b8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:09.945316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd185e3a-629b-4063-a3ae-764dedb0b8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:09.949073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd185e3a-629b-4063-a3ae-764dedb0b8ff service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:09.968904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 723/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:10.977701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a29eb-db91-4fe4-bd32-b5d590adac67 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:10.978603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a29eb-db91-4fe4-bd32-b5d590adac67 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:10.978603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a29eb-db91-4fe4-bd32-b5d590adac67 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:10.980351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 697/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:19:10.986173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631c59e-6083-43c3-a0e8-9b49df8cd5f8 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:10.986780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631c59e-6083-43c3-a0e8-9b49df8cd5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:10.986780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631c59e-6083-43c3-a0e8-9b49df8cd5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:11.007595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0631c59e-6083-43c3-a0e8-9b49df8cd5f8 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:11.009662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0631c59e-6083-43c3-a0e8-9b49df8cd5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:11.018150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 724/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:11.202544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a29eb-db91-4fe4-bd32-b5d590adac67 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:11.219228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a29eb-db91-4fe4-bd32-b5d590adac67 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:11.224989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1eb921-afc2-4f07-983a-ddc5450fac6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:11.232272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eb921-afc2-4f07-983a-ddc5450fac6d service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:11.232272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eb921-afc2-4f07-983a-ddc5450fac6d service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:11.232272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eb921-afc2-4f07-983a-ddc5450fac6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:11.235978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1eb921-afc2-4f07-983a-ddc5450fac6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:11.237901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1eb921-afc2-4f07-983a-ddc5450fac6d service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:11.251277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 698/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:12.237069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5ff9-4662-48a4-80ad-900943cd39f3 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:12.237687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5ff9-4662-48a4-80ad-900943cd39f3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:12.238052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5ff9-4662-48a4-80ad-900943cd39f3 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:12.240739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 725/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:12.246814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e12496-8b68-4199-bb4a-1894b54ea5b3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:12.247589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e12496-8b68-4199-bb4a-1894b54ea5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:12.247589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e12496-8b68-4199-bb4a-1894b54ea5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:12.270775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-97e12496-8b68-4199-bb4a-1894b54ea5b3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:12.272344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-97e12496-8b68-4199-bb4a-1894b54ea5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:12.284180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 699/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:12.441374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a5ff9-4662-48a4-80ad-900943cd39f3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:12.457729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a5ff9-4662-48a4-80ad-900943cd39f3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:12.463911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63209f6a-504c-4864-ab7b-e769e3056d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:12.464665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63209f6a-504c-4864-ab7b-e769e3056d25 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:12.464951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63209f6a-504c-4864-ab7b-e769e3056d25 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:12.465221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63209f6a-504c-4864-ab7b-e769e3056d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:12.474780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63209f6a-504c-4864-ab7b-e769e3056d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:12.476561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63209f6a-504c-4864-ab7b-e769e3056d25 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:12.491303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 726/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:13.522415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92858ed-f615-4135-991f-4bbf8b2d9025 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:13.523006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92858ed-f615-4135-991f-4bbf8b2d9025 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:13.523006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92858ed-f615-4135-991f-4bbf8b2d9025 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:13.525483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 700/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:13.531337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cf942-c5f9-4093-8304-a8740469ddc8 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:13.531934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cf942-c5f9-4093-8304-a8740469ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:13.532073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cf942-c5f9-4093-8304-a8740469ddc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:13.552102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6c3cf942-c5f9-4093-8304-a8740469ddc8 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:13.553408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6c3cf942-c5f9-4093-8304-a8740469ddc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:13.562342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 727/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:13.717819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92858ed-f615-4135-991f-4bbf8b2d9025 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:13.733641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92858ed-f615-4135-991f-4bbf8b2d9025 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:13.739521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e71ce3-03ec-4ac8-9185-686d4a67def0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:13.740203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e71ce3-03ec-4ac8-9185-686d4a67def0 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:13.740515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e71ce3-03ec-4ac8-9185-686d4a67def0 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:13.740797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e71ce3-03ec-4ac8-9185-686d4a67def0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:13.750993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e71ce3-03ec-4ac8-9185-686d4a67def0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:13.752701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e71ce3-03ec-4ac8-9185-686d4a67def0 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:13.764945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 701/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:14.776211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b0932-75f6-4447-bfef-662343420253 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:14.776858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b0932-75f6-4447-bfef-662343420253 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:14.776858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b0932-75f6-4447-bfef-662343420253 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:14.778260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 728/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:14.784626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d012-0a51-48de-9286-33d47a95e77e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:14.785478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d012-0a51-48de-9286-33d47a95e77e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:14.785478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d012-0a51-48de-9286-33d47a95e77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:14.805889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3529d012-0a51-48de-9286-33d47a95e77e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:14.807650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3529d012-0a51-48de-9286-33d47a95e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:14.818891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 702/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:14.973989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b0932-75f6-4447-bfef-662343420253 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:14.990428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b0932-75f6-4447-bfef-662343420253 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:14.996694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291567bd-15c4-4b30-bc23-eb80281c9ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:14.997406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291567bd-15c4-4b30-bc23-eb80281c9ff7 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:14.997740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291567bd-15c4-4b30-bc23-eb80281c9ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:14.998121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291567bd-15c4-4b30-bc23-eb80281c9ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:15.008120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291567bd-15c4-4b30-bc23-eb80281c9ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:15.009812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291567bd-15c4-4b30-bc23-eb80281c9ff7 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:15.021089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 729/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:15.998413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64985c66-eb64-4327-a906-9d41f08056bd None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:15.999278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64985c66-eb64-4327-a906-9d41f08056bd None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:15.999278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64985c66-eb64-4327-a906-9d41f08056bd None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:16.000446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 703/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:19:16.006616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ac6ed-cf61-4cda-a7c9-6dbc9aa1d92e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:16.007349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ac6ed-cf61-4cda-a7c9-6dbc9aa1d92e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:16.007349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ac6ed-cf61-4cda-a7c9-6dbc9aa1d92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:16.029371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-060ac6ed-cf61-4cda-a7c9-6dbc9aa1d92e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:16.032134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-060ac6ed-cf61-4cda-a7c9-6dbc9aa1d92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:16.042563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 730/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:16.223060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64985c66-eb64-4327-a906-9d41f08056bd None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:16.238286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64985c66-eb64-4327-a906-9d41f08056bd None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:16.244587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66abe-2953-470d-bf23-20dbe53f0bd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:16.245248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66abe-2953-470d-bf23-20dbe53f0bd9 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:16.245532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66abe-2953-470d-bf23-20dbe53f0bd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:16.245803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66abe-2953-470d-bf23-20dbe53f0bd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:16.255418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66abe-2953-470d-bf23-20dbe53f0bd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:16.257176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66abe-2953-470d-bf23-20dbe53f0bd9 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:16.268898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 704/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:17.243936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f260c-6191-47fc-a027-da9db842e640 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:17.244573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f260c-6191-47fc-a027-da9db842e640 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:17.244573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f260c-6191-47fc-a027-da9db842e640 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:17.246445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 731/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:19:17.252760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e486-f5e9-4cd1-bd60-de6636f203fc None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:17.253369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e486-f5e9-4cd1-bd60-de6636f203fc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:17.253504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e486-f5e9-4cd1-bd60-de6636f203fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:17.275653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7104e486-f5e9-4cd1-bd60-de6636f203fc None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:17.276982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7104e486-f5e9-4cd1-bd60-de6636f203fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:17.285589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 705/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:17.445433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f260c-6191-47fc-a027-da9db842e640 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:17.461352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f260c-6191-47fc-a027-da9db842e640 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:17.468110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddf342-f5e0-4125-b239-5fba3d6702ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:17.468829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddf342-f5e0-4125-b239-5fba3d6702ac service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:17.469113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddf342-f5e0-4125-b239-5fba3d6702ac service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:17.469391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddf342-f5e0-4125-b239-5fba3d6702ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:17.479849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddf342-f5e0-4125-b239-5fba3d6702ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:17.481653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddf342-f5e0-4125-b239-5fba3d6702ac service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:17.492537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 732/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:18.643760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cc1a8-d392-48c6-9f1e-1e8c7c4796a9 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:18.644473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cc1a8-d392-48c6-9f1e-1e8c7c4796a9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:18.644473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cc1a8-d392-48c6-9f1e-1e8c7c4796a9 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:18.646425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 706/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:18.654927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6abc3-47ca-4eb8-bd51-85d50d6a463f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:18.655655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6abc3-47ca-4eb8-bd51-85d50d6a463f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:18.655805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6abc3-47ca-4eb8-bd51-85d50d6a463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:18.686028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bcc6abc3-47ca-4eb8-bd51-85d50d6a463f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:18.687532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bcc6abc3-47ca-4eb8-bd51-85d50d6a463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:18.697633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 733/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:18.859049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cc1a8-d392-48c6-9f1e-1e8c7c4796a9 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:18.875164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cc1a8-d392-48c6-9f1e-1e8c7c4796a9 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:18.881160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d9d29-127a-4522-9ee9-580419b9a407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:18.881825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9d29-127a-4522-9ee9-580419b9a407 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:18.882103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9d29-127a-4522-9ee9-580419b9a407 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:18.882376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9d29-127a-4522-9ee9-580419b9a407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:18.893276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d9d29-127a-4522-9ee9-580419b9a407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:18.895026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d9d29-127a-4522-9ee9-580419b9a407 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:18.907268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 707/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:21.210168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d23e-05ee-4f2b-8498-e7991b4378a1 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:21.210956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d23e-05ee-4f2b-8498-e7991b4378a1 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:21.210956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d23e-05ee-4f2b-8498-e7991b4378a1 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:21.213071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 734/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:21.219672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a147e6f-a438-4dbb-a53f-267c66c85325 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:21.220335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a147e6f-a438-4dbb-a53f-267c66c85325 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:21.220472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a147e6f-a438-4dbb-a53f-267c66c85325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:21.241277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8a147e6f-a438-4dbb-a53f-267c66c85325 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:21.242891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8a147e6f-a438-4dbb-a53f-267c66c85325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:21.252309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 708/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:21.296229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d23e-05ee-4f2b-8498-e7991b4378a1 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:21.311912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d23e-05ee-4f2b-8498-e7991b4378a1 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:21.318370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e6708-2931-408b-a6bb-fc62a6f10881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:21.319470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e6708-2931-408b-a6bb-fc62a6f10881 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:21.319969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e6708-2931-408b-a6bb-fc62a6f10881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:21.320502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e6708-2931-408b-a6bb-fc62a6f10881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:21.332956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e6708-2931-408b-a6bb-fc62a6f10881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:21.334598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e6708-2931-408b-a6bb-fc62a6f10881 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:21.346965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 735/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:24.044875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353c2dd-485e-4098-ae71-d3b5c2435b61 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:24.045814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353c2dd-485e-4098-ae71-d3b5c2435b61 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:24.045814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353c2dd-485e-4098-ae71-d3b5c2435b61 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:24.046825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 709/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:19:24.054018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628235d-b1a5-405c-a37d-d922f882cd84 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:24.054654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628235d-b1a5-405c-a37d-d922f882cd84 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:24.054977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628235d-b1a5-405c-a37d-d922f882cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:24.080721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2628235d-b1a5-405c-a37d-d922f882cd84 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:24.082591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2628235d-b1a5-405c-a37d-d922f882cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:24.092478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 736/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:24.136595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353c2dd-485e-4098-ae71-d3b5c2435b61 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:24.151269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353c2dd-485e-4098-ae71-d3b5c2435b61 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:24.155509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5153b9-28c8-45ff-9aa4-3dea04c75597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:24.156169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5153b9-28c8-45ff-9aa4-3dea04c75597 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:24.156689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5153b9-28c8-45ff-9aa4-3dea04c75597 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:24.157002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5153b9-28c8-45ff-9aa4-3dea04c75597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:24.168772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5153b9-28c8-45ff-9aa4-3dea04c75597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:24.170553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5153b9-28c8-45ff-9aa4-3dea04c75597 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:24.184485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 710/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:27.164061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556d01-0c06-4f4f-bf50-f1b3af1bede6 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:27.164730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556d01-0c06-4f4f-bf50-f1b3af1bede6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:27.164730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556d01-0c06-4f4f-bf50-f1b3af1bede6 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:27.166029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 737/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:27.173674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cd917-b3ff-4bcb-b79b-6eb828f60c10 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:27.174264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cd917-b3ff-4bcb-b79b-6eb828f60c10 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:27.174633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cd917-b3ff-4bcb-b79b-6eb828f60c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:27.195752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-140cd917-b3ff-4bcb-b79b-6eb828f60c10 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:27.197175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-140cd917-b3ff-4bcb-b79b-6eb828f60c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:27.205434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 711/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:27.247187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556d01-0c06-4f4f-bf50-f1b3af1bede6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:27.259673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556d01-0c06-4f4f-bf50-f1b3af1bede6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:27.263769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137e9ce-c21b-4582-a714-7dc2b4fe8c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:27.264479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137e9ce-c21b-4582-a714-7dc2b4fe8c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:27.264766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137e9ce-c21b-4582-a714-7dc2b4fe8c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:27.265047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137e9ce-c21b-4582-a714-7dc2b4fe8c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:27.276717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137e9ce-c21b-4582-a714-7dc2b4fe8c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:27.278194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137e9ce-c21b-4582-a714-7dc2b4fe8c76 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:27.289584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 738/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:29.918049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9a140-ab1e-40a5-83a6-32a460553bac None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:29.918690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9a140-ab1e-40a5-83a6-32a460553bac None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:29.918690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9a140-ab1e-40a5-83a6-32a460553bac None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:29.920098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 712/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:29.928767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679a8c2-a76f-41d8-ab96-95dfa44cc813 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:29.930932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679a8c2-a76f-41d8-ab96-95dfa44cc813 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:29.930932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679a8c2-a76f-41d8-ab96-95dfa44cc813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:29.959640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c679a8c2-a76f-41d8-ab96-95dfa44cc813 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:29.961723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c679a8c2-a76f-41d8-ab96-95dfa44cc813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:29.975125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 739/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:30.017268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9a140-ab1e-40a5-83a6-32a460553bac None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:30.029387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9a140-ab1e-40a5-83a6-32a460553bac None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:30.033385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c1026-34cb-4b57-b8fd-6a531e22baf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:30.034079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1026-34cb-4b57-b8fd-6a531e22baf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:30.034409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1026-34cb-4b57-b8fd-6a531e22baf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:30.034721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1026-34cb-4b57-b8fd-6a531e22baf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:30.045488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c1026-34cb-4b57-b8fd-6a531e22baf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:30.046984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c1026-34cb-4b57-b8fd-6a531e22baf1 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:30.058590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 713/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:32.523987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087c91-adf5-4a61-98cd-adb244e6ef27 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:32.524625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087c91-adf5-4a61-98cd-adb244e6ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:32.524798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087c91-adf5-4a61-98cd-adb244e6ef27 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:32.528107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 740/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:32.535482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0ec9f-4e9c-42ad-a759-a664ed4bddc8 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:32.536043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0ec9f-4e9c-42ad-a759-a664ed4bddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:32.536510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0ec9f-4e9c-42ad-a759-a664ed4bddc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:32.565929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-79e0ec9f-4e9c-42ad-a759-a664ed4bddc8 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:32.567970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-79e0ec9f-4e9c-42ad-a759-a664ed4bddc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:32.582124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 714/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:32.638245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087c91-adf5-4a61-98cd-adb244e6ef27 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:32.652173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087c91-adf5-4a61-98cd-adb244e6ef27 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:32.657737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f86b4-01fd-491b-92a0-aae315afa067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:32.657737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f86b4-01fd-491b-92a0-aae315afa067 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:32.658076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f86b4-01fd-491b-92a0-aae315afa067 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:32.658328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f86b4-01fd-491b-92a0-aae315afa067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:32.669527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f86b4-01fd-491b-92a0-aae315afa067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:32.671003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f86b4-01fd-491b-92a0-aae315afa067 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:32.682411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 741/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:35.159412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb621-ecc1-4e29-a3bb-957d304734d0 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:35.160004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb621-ecc1-4e29-a3bb-957d304734d0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:35.160405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb621-ecc1-4e29-a3bb-957d304734d0 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:35.163152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 715/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:35.170655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536e0d7-ddb1-4491-94a9-28329fdc9a7e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:35.171252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536e0d7-ddb1-4491-94a9-28329fdc9a7e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:35.171252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536e0d7-ddb1-4491-94a9-28329fdc9a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:35.177682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=70 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:19:35.191694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2536e0d7-ddb1-4491-94a9-28329fdc9a7e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:35.192995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2536e0d7-ddb1-4491-94a9-28329fdc9a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:35.202733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 742/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:35.267387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fb621-ecc1-4e29-a3bb-957d304734d0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:35.274001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=64 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:19:35.281208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fb621-ecc1-4e29-a3bb-957d304734d0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:35.285100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197cda0-b067-4024-ae3d-fb82cff585fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:35.285772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197cda0-b067-4024-ae3d-fb82cff585fd service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:35.286056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197cda0-b067-4024-ae3d-fb82cff585fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:35.286333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197cda0-b067-4024-ae3d-fb82cff585fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:35.297158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197cda0-b067-4024-ae3d-fb82cff585fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:35.298678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197cda0-b067-4024-ae3d-fb82cff585fd service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:35.310775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 716/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:37.773893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1523ba-5ab5-481e-befa-f5d13b0847e7 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:37.774827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1523ba-5ab5-481e-befa-f5d13b0847e7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:37.774827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1523ba-5ab5-481e-befa-f5d13b0847e7 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:37.776792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 743/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:19:37.782994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c84e9-528e-47f9-b456-c653d7f47e1d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:37.783744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c84e9-528e-47f9-b456-c653d7f47e1d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:37.783744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c84e9-528e-47f9-b456-c653d7f47e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:37.802081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a47c84e9-528e-47f9-b456-c653d7f47e1d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:37.803303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a47c84e9-528e-47f9-b456-c653d7f47e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:37.811365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 717/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:37.850349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1523ba-5ab5-481e-befa-f5d13b0847e7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:37.861906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1523ba-5ab5-481e-befa-f5d13b0847e7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:37.865454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca620828-007a-4bd5-90d9-277ffa8fc17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:37.866104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca620828-007a-4bd5-90d9-277ffa8fc17a service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:37.866384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca620828-007a-4bd5-90d9-277ffa8fc17a service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:37.866656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca620828-007a-4bd5-90d9-277ffa8fc17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:37.877006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca620828-007a-4bd5-90d9-277ffa8fc17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:37.878882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca620828-007a-4bd5-90d9-277ffa8fc17a service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:37.897422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 744/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:40.356396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966e004-9d25-4bfa-b383-d8d9ba196981 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:40.356982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966e004-9d25-4bfa-b383-d8d9ba196981 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:40.357393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966e004-9d25-4bfa-b383-d8d9ba196981 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:40.359994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 718/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:40.366281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585eef5-a86b-4634-848f-107ab3e60586 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:40.366874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585eef5-a86b-4634-848f-107ab3e60586 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:40.366874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585eef5-a86b-4634-848f-107ab3e60586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:40.386629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0585eef5-a86b-4634-848f-107ab3e60586 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:40.388019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0585eef5-a86b-4634-848f-107ab3e60586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:40.396937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 745/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:19:40.436353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966e004-9d25-4bfa-b383-d8d9ba196981 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:40.448320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966e004-9d25-4bfa-b383-d8d9ba196981 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:40.452038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce139a3-5a6d-47e3-9cbd-9a862b279fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:40.452755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce139a3-5a6d-47e3-9cbd-9a862b279fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:40.453042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce139a3-5a6d-47e3-9cbd-9a862b279fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:40.453299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce139a3-5a6d-47e3-9cbd-9a862b279fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:40.463246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce139a3-5a6d-47e3-9cbd-9a862b279fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:40.464779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce139a3-5a6d-47e3-9cbd-9a862b279fa2 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:40.475484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 719/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:42.959709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df84062-35bc-4f02-ab62-c82e54b9c90c None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:42.960474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df84062-35bc-4f02-ab62-c82e54b9c90c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:42.960474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df84062-35bc-4f02-ab62-c82e54b9c90c None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:42.961721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 746/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:42.968395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6967c-e78a-42fe-bd10-8cb6d8b7a6a3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:42.969221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6967c-e78a-42fe-bd10-8cb6d8b7a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:42.969221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6967c-e78a-42fe-bd10-8cb6d8b7a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:42.988904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fcc6967c-e78a-42fe-bd10-8cb6d8b7a6a3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:42.990304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fcc6967c-e78a-42fe-bd10-8cb6d8b7a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:42.999980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 720/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:43.039120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df84062-35bc-4f02-ab62-c82e54b9c90c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:43.050893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df84062-35bc-4f02-ab62-c82e54b9c90c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:43.054722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35c6c-2695-4d66-9a8a-e7281823b5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:43.055377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35c6c-2695-4d66-9a8a-e7281823b5ab service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:43.055672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35c6c-2695-4d66-9a8a-e7281823b5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:43.055944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35c6c-2695-4d66-9a8a-e7281823b5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:43.071488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35c6c-2695-4d66-9a8a-e7281823b5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:43.073800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35c6c-2695-4d66-9a8a-e7281823b5ab service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:43.085519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 747/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:19:45.490416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca30aa8-e747-490d-9b9c-7c02c9633c0d None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:45.491014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca30aa8-e747-490d-9b9c-7c02c9633c0d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:45.491014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca30aa8-e747-490d-9b9c-7c02c9633c0d None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:45.492555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 721/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:19:45.498788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2d96e-a250-450b-9e1d-96629186f1bb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:45.499319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2d96e-a250-450b-9e1d-96629186f1bb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:45.499319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2d96e-a250-450b-9e1d-96629186f1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:45.517483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-15e2d96e-a250-450b-9e1d-96629186f1bb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:19:45.518717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-15e2d96e-a250-450b-9e1d-96629186f1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:19:45.527458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 748/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:19:45.566232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca30aa8-e747-490d-9b9c-7c02c9633c0d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:19:45.578209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca30aa8-e747-490d-9b9c-7c02c9633c0d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:19:45.582150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a888e3-06ac-4679-86c9-0b685cb33a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:19:45.582919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a888e3-06ac-4679-86c9-0b685cb33a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:19:45.583298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a888e3-06ac-4679-86c9-0b685cb33a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:19:45.583683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a888e3-06ac-4679-86c9-0b685cb33a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:19:45.594823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a888e3-06ac-4679-86c9-0b685cb33a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:19:45.596445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a888e3-06ac-4679-86c9-0b685cb33a39 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:19:45.607692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 722/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:19:55.511889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=20 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:19:55.602460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=23 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:20:19.763590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f8baa-9da1-4886-a42a-a6d046c351cc None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:19.764189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f8baa-9da1-4886-a42a-a6d046c351cc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:19.764189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f8baa-9da1-4886-a42a-a6d046c351cc None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:19.765751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 749/1471] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:19.771414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3a3d-c2ab-4438-ac1e-7a8e4ce625f7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:19.771957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3a3d-c2ab-4438-ac1e-7a8e4ce625f7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:19.772062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3a3d-c2ab-4438-ac1e-7a8e4ce625f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:19.791324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e51d3a3d-c2ab-4438-ac1e-7a8e4ce625f7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:19.793123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e51d3a3d-c2ab-4438-ac1e-7a8e4ce625f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:19.803016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 723/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:19.855411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a7d80-5df1-4af6-92f8-0fda72c8ed26 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:19.855691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a7d80-5df1-4af6-92f8-0fda72c8ed26 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:19.855965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a7d80-5df1-4af6-92f8-0fda72c8ed26 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:19.857385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 750/1473] 10.222.0.46 () {54 vars in 962 bytes} [Sat Jun 4 09:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:19.863190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a99782-8ea5-4f2f-b5af-3605ca1a95e2 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:19.863489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a99782-8ea5-4f2f-b5af-3605ca1a95e2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:19.863901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a99782-8ea5-4f2f-b5af-3605ca1a95e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:19.882863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-66a99782-8ea5-4f2f-b5af-3605ca1a95e2 None None] MFA Rules not processed for user `8b3a9e1e2e3d438f92286a4158c8d4cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:19.884077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-66a99782-8ea5-4f2f-b5af-3605ca1a95e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:19.890922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 724/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Sat Jun 4 09:20:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:19.898559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a7d80-5df1-4af6-92f8-0fda72c8ed26 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:19.909894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a7d80-5df1-4af6-92f8-0fda72c8ed26 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:19.913742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b891a02-da1c-4bc2-b0f3-b1d3a19ecfcb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:19.914391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891a02-da1c-4bc2-b0f3-b1d3a19ecfcb service glance] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:19.914676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891a02-da1c-4bc2-b0f3-b1d3a19ecfcb service glance] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:19.914958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891a02-da1c-4bc2-b0f3-b1d3a19ecfcb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:19.926805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891a02-da1c-4bc2-b0f3-b1d3a19ecfcb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:19.928531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891a02-da1c-4bc2-b0f3-b1d3a19ecfcb service glance] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:19.941437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 751/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:20:22.408500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82abb1-b8ac-4047-b29c-c536f6736af8 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:22.409103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82abb1-b8ac-4047-b29c-c536f6736af8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:22.409103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82abb1-b8ac-4047-b29c-c536f6736af8 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:22.410409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 725/1476] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:22.416236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40c02-69df-4655-a648-fec61af8c9db None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:22.416802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40c02-69df-4655-a648-fec61af8c9db None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:22.416802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40c02-69df-4655-a648-fec61af8c9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:22.436102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c7b40c02-69df-4655-a648-fec61af8c9db None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:22.437349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c7b40c02-69df-4655-a648-fec61af8c9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:22.445665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 752/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:22.470466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82abb1-b8ac-4047-b29c-c536f6736af8 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:22.485450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82abb1-b8ac-4047-b29c-c536f6736af8 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:22.491301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:22.491969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:22.492251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:22.492560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:22.493402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:22.495785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:20:22.495785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:20:22.495785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:20:22.496966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0f4f8-9949-4255-b39e-08eee5b7ac57 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:22.517996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 726/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Sat Jun 4 09:20:22 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:20:24.917024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9352364-8369-4543-b3e1-5b79269b6135 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:24.917643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9352364-8369-4543-b3e1-5b79269b6135 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:24.917643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9352364-8369-4543-b3e1-5b79269b6135 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:24.918961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 753/1479] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:24.924446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3687ae2-0331-4738-a82d-ee16309132f5 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:24.924943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3687ae2-0331-4738-a82d-ee16309132f5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:24.925065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3687ae2-0331-4738-a82d-ee16309132f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:24.945632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a3687ae2-0331-4738-a82d-ee16309132f5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:24.946862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a3687ae2-0331-4738-a82d-ee16309132f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:24.955771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 727/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:25.001911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9352364-8369-4543-b3e1-5b79269b6135 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:25.017591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9352364-8369-4543-b3e1-5b79269b6135 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:25.023046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2516ccf-0672-48c1-a979-d47e397e1204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:25.023758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2516ccf-0672-48c1-a979-d47e397e1204 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:25.024013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2516ccf-0672-48c1-a979-d47e397e1204 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:25.024298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2516ccf-0672-48c1-a979-d47e397e1204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:25.034620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2516ccf-0672-48c1-a979-d47e397e1204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:25.036232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2516ccf-0672-48c1-a979-d47e397e1204 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:25.047889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 754/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:20:27.830352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb43bd4-40b8-4875-be79-f8079568dfa2 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:27.830969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb43bd4-40b8-4875-be79-f8079568dfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:27.830969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb43bd4-40b8-4875-be79-f8079568dfa2 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:27.832475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 728/1482] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:27.837792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334e3f6-0055-4f8b-aade-de2a5141bc18 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:27.838215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334e3f6-0055-4f8b-aade-de2a5141bc18 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:27.838349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334e3f6-0055-4f8b-aade-de2a5141bc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:27.858038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5334e3f6-0055-4f8b-aade-de2a5141bc18 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:27.859251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5334e3f6-0055-4f8b-aade-de2a5141bc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:27.867171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 755/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:27.904515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb43bd4-40b8-4875-be79-f8079568dfa2 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:27.915702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb43bd4-40b8-4875-be79-f8079568dfa2 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:27.919796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492c525-7435-4f7a-a4fa-2af7b04261e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:27.920548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c525-7435-4f7a-a4fa-2af7b04261e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:27.920857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c525-7435-4f7a-a4fa-2af7b04261e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:27.921131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c525-7435-4f7a-a4fa-2af7b04261e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:27.931031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492c525-7435-4f7a-a4fa-2af7b04261e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:27.932614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492c525-7435-4f7a-a4fa-2af7b04261e6 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:27.943302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 729/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:20:30.325796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5c488-ac69-4fda-a16c-1ba762712ea2 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:30.326368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5c488-ac69-4fda-a16c-1ba762712ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:30.326368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5c488-ac69-4fda-a16c-1ba762712ea2 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:30.327763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 756/1485] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:30.333539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7f82-d360-4f08-b442-05570cdf0786 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:30.334041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7f82-d360-4f08-b442-05570cdf0786 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:30.334153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7f82-d360-4f08-b442-05570cdf0786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:30.354020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5a9f7f82-d360-4f08-b442-05570cdf0786 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:30.355249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5a9f7f82-d360-4f08-b442-05570cdf0786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:30.363274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 730/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:30.402696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5c488-ac69-4fda-a16c-1ba762712ea2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:30.415863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5c488-ac69-4fda-a16c-1ba762712ea2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:30.419770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6795c4e-a1c4-4f24-9ac3-b64d652cb79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:30.420463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6795c4e-a1c4-4f24-9ac3-b64d652cb79f service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:30.420744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6795c4e-a1c4-4f24-9ac3-b64d652cb79f service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:30.421017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6795c4e-a1c4-4f24-9ac3-b64d652cb79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:30.430862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6795c4e-a1c4-4f24-9ac3-b64d652cb79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:30.432311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6795c4e-a1c4-4f24-9ac3-b64d652cb79f service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:30.444028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 757/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:20:32.948181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eedc63-c06e-427b-91bb-092abfbc2467 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:32.949142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eedc63-c06e-427b-91bb-092abfbc2467 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:32.949142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eedc63-c06e-427b-91bb-092abfbc2467 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:32.950232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 731/1488] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:20:32.956603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dc02d-70a9-4016-bc37-b0ee4f54cb0c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:32.957252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dc02d-70a9-4016-bc37-b0ee4f54cb0c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:32.957252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dc02d-70a9-4016-bc37-b0ee4f54cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:32.977224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-747dc02d-70a9-4016-bc37-b0ee4f54cb0c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:32.978481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-747dc02d-70a9-4016-bc37-b0ee4f54cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:32.987584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 758/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:33.030647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eedc63-c06e-427b-91bb-092abfbc2467 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:33.043199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eedc63-c06e-427b-91bb-092abfbc2467 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:33.047152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1167d0-447d-4584-9275-4876956b897f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:33.047810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1167d0-447d-4584-9275-4876956b897f service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:33.048100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1167d0-447d-4584-9275-4876956b897f service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:33.048426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1167d0-447d-4584-9275-4876956b897f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:33.058813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1167d0-447d-4584-9275-4876956b897f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:33.060476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1167d0-447d-4584-9275-4876956b897f service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:33.071575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 732/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:20:35.993076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab22a7-97f7-4ed6-9011-9f61f7acff3c None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:35.993943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab22a7-97f7-4ed6-9011-9f61f7acff3c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:35.994121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab22a7-97f7-4ed6-9011-9f61f7acff3c None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:35.995751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 759/1491] 10.222.0.46 () {54 vars in 926 bytes} [Sat Jun 4 09:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:20:36.002767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc1045-812f-468a-91d3-ae2718de0e93 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:36.003435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc1045-812f-468a-91d3-ae2718de0e93 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:36.003920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc1045-812f-468a-91d3-ae2718de0e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:36.034858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9afc1045-812f-468a-91d3-ae2718de0e93 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:20:36.036992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9afc1045-812f-468a-91d3-ae2718de0e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:20:36.051154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 733/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Sat Jun 4 09:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:20:36.092495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab22a7-97f7-4ed6-9011-9f61f7acff3c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:20:36.104108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab22a7-97f7-4ed6-9011-9f61f7acff3c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:20:36.108563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193f84-8a5a-45f3-b96e-e70b847315cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:20:36.109230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193f84-8a5a-45f3-b96e-e70b847315cd service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:20:36.109512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193f84-8a5a-45f3-b96e-e70b847315cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:20:36.109792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193f84-8a5a-45f3-b96e-e70b847315cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:20:36.125610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193f84-8a5a-45f3-b96e-e70b847315cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:20:36.128274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193f84-8a5a-45f3-b96e-e70b847315cd service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:20:36.146772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 760/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:20:46.025451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=33 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:20:46.138414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=33 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:22:21.534188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:21.535608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:21.535608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:21.554513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:22:21.555747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:21.563870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 734/1494] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:22:21.572974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:21.573738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:21.573738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:21.594119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:22:21.595457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:21.603868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 761/1495] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:22:21.609554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:21.619820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02765d-1095-4f40-a3a7-8989e5993002 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:21.624663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:21.625347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:21.625629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:21.625903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:21.626645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:22:21.626955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:21.627999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:21.627999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:22:21.627999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:22:21.628928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:21.642337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 735/1496] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:22:21.651109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:21.665470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362ef7-cb14-4cbe-a969-11183dd36373 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:21.670955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:21.671628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:21.671907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:21.672182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:21.672914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:21.674715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:21.674715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:22:21.674715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:22:21.675905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:21.770291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 762/1497] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:22:21 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:22:21.774964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:21.786311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78997-1bed-4839-8eb1-e04a13025fe2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:21.848783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:21.849468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:21.849756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:21.850036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:21.850752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:21.851861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:21.851861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:22:21.851861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:22:21.853173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:21.974813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 736/1498] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:22:21 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:22:21.978933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:21.989512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fe5b2-f1c4-4f62-abe0-9727be23d0a8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:21.994284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:21.994970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:21.995246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:21.995517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:21.996252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:21.997316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:21.997316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:22:21.997316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:22:21.998198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:22.011925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 763/1499] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:22:22.016968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:22.027606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc210da-8f41-426e-b3f6-45a6b0577c7f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:22.033323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:22.034014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.034300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.034589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.035324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:22.036465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:22.036465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:22:22.036465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:22:22.037283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1e5f2-7d85-461f-bd52-aa6a206e7851 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:22.052568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 737/1500] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:22:22.056979 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:22.068316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918bcf8-fb46-4d0e-870a-4f62b169408d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:22.073541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:22.074474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.074791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.075065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] PATH_INFO: `/v3/projects/6054ae56c79649a6b51df5103c4f0580/users/9961205120d04f70b1a656a3ccd2de92/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.101201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6054ae56c79649a6b51df5103c4f0580, user_id=9961205120d04f70b1a656a3ccd2de92, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:22.103110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:22.103110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:22:22.103110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:22:22.106276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db543bf-e036-49fb-95bd-f412873fda7b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:22.323015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 764/1501] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:22:22 2022] PUT /identity/v3/projects/6054ae56c79649a6b51df5103c4f0580/users/9961205120d04f70b1a656a3ccd2de92/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:22:22.336191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debfd55-d94e-4cf2-bc34-a5f1ced08269 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.336512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debfd55-d94e-4cf2-bc34-a5f1ced08269 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.336798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debfd55-d94e-4cf2-bc34-a5f1ced08269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.365447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5debfd55-d94e-4cf2-bc34-a5f1ced08269 None None] MFA Rules not processed for user `9961205120d04f70b1a656a3ccd2de92`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:22:22.366691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5debfd55-d94e-4cf2-bc34-a5f1ced08269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:22.439303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 738/1502] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:22:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:22:22.454069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78074-f928-4f7d-801e-f17ae6686f95 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.454355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78074-f928-4f7d-801e-f17ae6686f95 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.454690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78074-f928-4f7d-801e-f17ae6686f95 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.456144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 765/1503] 10.222.0.46 () {54 vars in 963 bytes} [Sat Jun 4 09:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:22:22.469181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd950f-f015-45fc-9cf2-8b1e058ed4fc None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.469473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd950f-f015-45fc-9cf2-8b1e058ed4fc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.469760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd950f-f015-45fc-9cf2-8b1e058ed4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.489349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3dfd950f-f015-45fc-9cf2-8b1e058ed4fc None None] MFA Rules not processed for user `e53e659f7faf403b9f33f393d166dda3`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:22:22.490572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3dfd950f-f015-45fc-9cf2-8b1e058ed4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:22.546328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:22:22.555678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 739/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Sat Jun 4 09:22:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:22:22.564259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78074-f928-4f7d-801e-f17ae6686f95 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:22.575759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78074-f928-4f7d-801e-f17ae6686f95 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:22.580489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:22.582329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.582329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.582329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.591180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:22.592711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:22.602899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 766/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:22:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:22:22.659840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317eb1-018c-4a39-bea7-47848e9dbb30 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.660403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317eb1-018c-4a39-bea7-47848e9dbb30 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.660891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317eb1-018c-4a39-bea7-47848e9dbb30 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.663549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 740/1506] 10.222.0.46 () {52 vars in 829 bytes} [Sat Jun 4 09:22:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:22:22.669016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:22.758670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2961f39-5788-4fb8-a2e7-7daa53fc736a service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:22.763742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:22.764743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:22.765033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:22.765312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] PATH_INFO: `/v3/users/9961205120d04f70b1a656a3ccd2de92` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:22.766571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9961205120d04f70b1a656a3ccd2de92)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:22.767870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:22.767870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:22:22.767870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:22:22.769278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:23.044965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9961205120d04f70b1a656a3ccd2de92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:23.061038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9961205120d04f70b1a656a3ccd2de92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:23.062103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9961205120d04f70b1a656a3ccd2de92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:23.082531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9961205120d04f70b1a656a3ccd2de92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:23.083311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9961205120d04f70b1a656a3ccd2de92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:23.087859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:22:23.208123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 767/1507] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:22:22 2022] DELETE /identity/v3/users/9961205120d04f70b1a656a3ccd2de92 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:22:23.213199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:23.213635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:23.214022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:23.233911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:22:23.235159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:23.307708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 741/1508] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:22:23.319902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:23.321845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:23.425295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c20d-e0f7-4607-8c8a-2225986e37cd admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:23.431602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219024f7-e233-4659-97ff-393520a253fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:23.432779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219024f7-e233-4659-97ff-393520a253fb service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:23.433294 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219024f7-e233-4659-97ff-393520a253fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:23.433804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219024f7-e233-4659-97ff-393520a253fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:23.451808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219024f7-e233-4659-97ff-393520a253fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:23.454865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219024f7-e233-4659-97ff-393520a253fb service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:23.474293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 768/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:22:24.293124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:22:24.295944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:22:24.309522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ccb4a-33fd-4753-8716-178266d0b6ab None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:22:24.315897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:22:24.316426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:22:24.316713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:22:24.317003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] PATH_INFO: `/v3/projects/6054ae56c79649a6b51df5103c4f0580` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:22:24.318416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6054ae56c79649a6b51df5103c4f0580)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:22:24.319481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:22:24.319481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:22:24.319481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:22:24.320807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:22:24.506298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6054ae56c79649a6b51df5103c4f0580', 'request_id': 'req-21a4c65b-57c6-461e-843c-a38aaafa4f5f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:24.521943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6054ae56c79649a6b51df5103c4f0580', 'request_id': 'req-21a4c65b-57c6-461e-843c-a38aaafa4f5f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:24.596365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6054ae56c79649a6b51df5103c4f0580', 'request_id': 'req-21a4c65b-57c6-461e-843c-a38aaafa4f5f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:24.610901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] The token cache is being invalidate because project 6054ae56c79649a6b51df5103c4f0580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:22:24.611184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:24.611985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-21a4c65b-57c6-461e-843c-a38aaafa4f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:22:24.613657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 742/1510] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:22:24 2022] DELETE /identity/v3/projects/6054ae56c79649a6b51df5103c4f0580 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:22:33.465563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:22:34.606012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:24:00.426383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39080c-70b1-472b-96d1-dce15e6a62c5 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:00.427041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39080c-70b1-472b-96d1-dce15e6a62c5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:00.427041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39080c-70b1-472b-96d1-dce15e6a62c5 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:00.428450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 769/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:24:00.434393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678d047-104c-489c-be56-47aa2e507a6e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:00.435079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678d047-104c-489c-be56-47aa2e507a6e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:00.435079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678d047-104c-489c-be56-47aa2e507a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:00.456661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d678d047-104c-489c-be56-47aa2e507a6e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:00.457941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d678d047-104c-489c-be56-47aa2e507a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:00.534178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 743/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:00.713607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39080c-70b1-472b-96d1-dce15e6a62c5 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:00.715356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4a39080c-70b1-472b-96d1-dce15e6a62c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:00.821739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39080c-70b1-472b-96d1-dce15e6a62c5 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:00.831230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e197c94-5e3f-4092-9489-2d2c8f78dfc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:00.832381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e197c94-5e3f-4092-9489-2d2c8f78dfc7 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:00.832875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e197c94-5e3f-4092-9489-2d2c8f78dfc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:00.833368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e197c94-5e3f-4092-9489-2d2c8f78dfc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:00.848818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e197c94-5e3f-4092-9489-2d2c8f78dfc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:00.851618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e197c94-5e3f-4092-9489-2d2c8f78dfc7 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:00.870023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 770/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:24:01.579826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a9b24-7c4d-4086-ad0a-5b5ea7b3561b None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:01.580723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a9b24-7c4d-4086-ad0a-5b5ea7b3561b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:01.580942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a9b24-7c4d-4086-ad0a-5b5ea7b3561b None None] PATH_INFO: `/v3` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:01.582733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 744/1514] 10.222.0.47 () {54 vars in 925 bytes} [Sat Jun 4 09:24:01 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:24:01.590787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c6eed-0878-4b1c-9606-d531a875a863 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:01.591373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c6eed-0878-4b1c-9606-d531a875a863 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:01.592029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c6eed-0878-4b1c-9606-d531a875a863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:01.627905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-471c6eed-0878-4b1c-9606-d531a875a863 None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:01.630176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-471c6eed-0878-4b1c-9606-d531a875a863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:01.753506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 771/1515] 10.222.0.47 () {58 vars in 1015 bytes} [Sat Jun 4 09:24:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:01.882785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831594b-4a32-4a6e-b5b4-6009f8a26f79 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:01.883190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831594b-4a32-4a6e-b5b4-6009f8a26f79 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:01.883686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831594b-4a32-4a6e-b5b4-6009f8a26f79 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:01.886320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 745/1516] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:24:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:24:01.893414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b048f04-7575-4d20-99db-0af60824c5c8 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:01.893911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b048f04-7575-4d20-99db-0af60824c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:01.894398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b048f04-7575-4d20-99db-0af60824c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:01.919641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3b048f04-7575-4d20-99db-0af60824c5c8 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:01.920935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b048f04-7575-4d20-99db-0af60824c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:01.930113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 772/1517] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:02.087884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831594b-4a32-4a6e-b5b4-6009f8a26f79 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:02.106699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831594b-4a32-4a6e-b5b4-6009f8a26f79 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:02.113937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0971-7993-4e5e-9181-11d93f9e5c43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:02.114825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0971-7993-4e5e-9181-11d93f9e5c43 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:02.115201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0971-7993-4e5e-9181-11d93f9e5c43 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:02.115488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0971-7993-4e5e-9181-11d93f9e5c43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:02.127842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0971-7993-4e5e-9181-11d93f9e5c43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:02.130178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0971-7993-4e5e-9181-11d93f9e5c43 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:02.142675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 746/1518] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:24:10.832647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b13d-7996-48e0-ba6e-c831a9d12fd9 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:10.833227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b13d-7996-48e0-ba6e-c831a9d12fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:10.833227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b13d-7996-48e0-ba6e-c831a9d12fd9 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:10.834631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 773/1519] 10.222.0.46 () {54 vars in 960 bytes} [Sat Jun 4 09:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:24:10.840377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55257a2-6a35-4c2e-9246-075aa8031f63 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:10.841228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55257a2-6a35-4c2e-9246-075aa8031f63 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:10.841228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55257a2-6a35-4c2e-9246-075aa8031f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:10.860968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e55257a2-6a35-4c2e-9246-075aa8031f63 None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:10.862269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e55257a2-6a35-4c2e-9246-075aa8031f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:10.871616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 747/1520] 10.222.0.46 () {58 vars in 1057 bytes} [Sat Jun 4 09:24:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:24:10.880259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b13d-7996-48e0-ba6e-c831a9d12fd9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:10.895076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b13d-7996-48e0-ba6e-c831a9d12fd9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:10.900789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffb098-4df7-4118-a8b3-09cc6831ab6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:10.901554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb098-4df7-4118-a8b3-09cc6831ab6e service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:10.901884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb098-4df7-4118-a8b3-09cc6831ab6e service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:10.902204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb098-4df7-4118-a8b3-09cc6831ab6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:10.912542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffb098-4df7-4118-a8b3-09cc6831ab6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:10.914345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffb098-4df7-4118-a8b3-09cc6831ab6e service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:10.925998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 774/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:24:13.510658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55919c5c-884f-422c-940d-66465273b637 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:13.511217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55919c5c-884f-422c-940d-66465273b637 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:13.511370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55919c5c-884f-422c-940d-66465273b637 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:13.512946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 748/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:24:13.521747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e489e-d62a-43b6-bb61-fde2ffde26c7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:13.522341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e489e-d62a-43b6-bb61-fde2ffde26c7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:13.522707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e489e-d62a-43b6-bb61-fde2ffde26c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:13.555734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2f7e489e-d62a-43b6-bb61-fde2ffde26c7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:13.557663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2f7e489e-d62a-43b6-bb61-fde2ffde26c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:13.572115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 775/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:13.742445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55919c5c-884f-422c-940d-66465273b637 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:13.767226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55919c5c-884f-422c-940d-66465273b637 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:13.776311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbff9d9-0b3a-435b-aad2-a478f8263e2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:13.777415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbff9d9-0b3a-435b-aad2-a478f8263e2a service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:13.777949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbff9d9-0b3a-435b-aad2-a478f8263e2a service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:13.778374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbff9d9-0b3a-435b-aad2-a478f8263e2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:13.794906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbff9d9-0b3a-435b-aad2-a478f8263e2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:13.797722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbff9d9-0b3a-435b-aad2-a478f8263e2a service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:13.815832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 749/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:24:14.772708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34151c-a901-4a95-a64c-943c87f2e881 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:14.773306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34151c-a901-4a95-a64c-943c87f2e881 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:14.773617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34151c-a901-4a95-a64c-943c87f2e881 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:14.775303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 776/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 4 09:24:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:24:14.781582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedbd8-3c49-45d3-802c-1b31862491b7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:14.782156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedbd8-3c49-45d3-802c-1b31862491b7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:14.782295 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedbd8-3c49-45d3-802c-1b31862491b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:14.806509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-73aedbd8-3c49-45d3-802c-1b31862491b7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:14.808139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-73aedbd8-3c49-45d3-802c-1b31862491b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:14.819464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 750/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 4 09:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:14.982727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34151c-a901-4a95-a64c-943c87f2e881 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:14.998675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34151c-a901-4a95-a64c-943c87f2e881 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:15.005212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fe777-31cb-4cab-b093-5012100f3148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:15.005922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fe777-31cb-4cab-b093-5012100f3148 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:15.006248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fe777-31cb-4cab-b093-5012100f3148 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:15.006569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fe777-31cb-4cab-b093-5012100f3148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:15.016525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fe777-31cb-4cab-b093-5012100f3148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:15.018388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fe777-31cb-4cab-b093-5012100f3148 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:15.029552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 777/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:24:19.417997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f070-efa8-4c46-9e39-c2472c9c453f None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:19.418702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f070-efa8-4c46-9e39-c2472c9c453f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:19.418702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f070-efa8-4c46-9e39-c2472c9c453f None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:19.419968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 751/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:24:19.426082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a99b8-e176-4176-a66d-eb02c366144f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:19.426725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a99b8-e176-4176-a66d-eb02c366144f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:19.426855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a99b8-e176-4176-a66d-eb02c366144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:19.447146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ed9a99b8-e176-4176-a66d-eb02c366144f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:19.448424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ed9a99b8-e176-4176-a66d-eb02c366144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:19.457335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 778/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:24:19.496033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716f070-efa8-4c46-9e39-c2472c9c453f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:19.497854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d716f070-efa8-4c46-9e39-c2472c9c453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:19.565838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716f070-efa8-4c46-9e39-c2472c9c453f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:19.569649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc3148-3e1a-4c8a-8111-5394883fa295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:19.570323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc3148-3e1a-4c8a-8111-5394883fa295 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:19.570609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc3148-3e1a-4c8a-8111-5394883fa295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:19.570895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc3148-3e1a-4c8a-8111-5394883fa295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:19.582916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc3148-3e1a-4c8a-8111-5394883fa295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:19.583800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc3148-3e1a-4c8a-8111-5394883fa295 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:19.594839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 752/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:24:21.986653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67140ec5-2c7f-4428-9b03-fc7e886b588b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:21.987250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67140ec5-2c7f-4428-9b03-fc7e886b588b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:21.987250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67140ec5-2c7f-4428-9b03-fc7e886b588b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:21.989408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 779/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:24:21.995514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d063d1a-8a1e-421e-9dbb-efba6c93dec9 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:21.996148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d063d1a-8a1e-421e-9dbb-efba6c93dec9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:21.996148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d063d1a-8a1e-421e-9dbb-efba6c93dec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:22.015771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0d063d1a-8a1e-421e-9dbb-efba6c93dec9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:22.017111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0d063d1a-8a1e-421e-9dbb-efba6c93dec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:22.025448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 753/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:22.066529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67140ec5-2c7f-4428-9b03-fc7e886b588b None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:22.068516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-67140ec5-2c7f-4428-9b03-fc7e886b588b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:22.082671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67140ec5-2c7f-4428-9b03-fc7e886b588b None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:22.087545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeb0e9-be73-4047-b570-fb147b84b982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:22.088681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeb0e9-be73-4047-b570-fb147b84b982 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:22.089188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeb0e9-be73-4047-b570-fb147b84b982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:22.089828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeb0e9-be73-4047-b570-fb147b84b982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:22.103302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeb0e9-be73-4047-b570-fb147b84b982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:22.104994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeb0e9-be73-4047-b570-fb147b84b982 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:22.117065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 780/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:24:25.995039 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd72465-bde7-405e-926e-5a86ba0c8f26 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:25.996007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd72465-bde7-405e-926e-5a86ba0c8f26 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:25.996007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd72465-bde7-405e-926e-5a86ba0c8f26 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:25.997303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 754/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:24:26.004514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e507cc-55e4-4fb6-98df-63452ec788d1 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:26.005419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e507cc-55e4-4fb6-98df-63452ec788d1 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:26.005607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e507cc-55e4-4fb6-98df-63452ec788d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:26.013060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:24:26.027821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-45e507cc-55e4-4fb6-98df-63452ec788d1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:26.029327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-45e507cc-55e4-4fb6-98df-63452ec788d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:26.038278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 781/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:24:26.326309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd72465-bde7-405e-926e-5a86ba0c8f26 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:26.328220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fdd72465-bde7-405e-926e-5a86ba0c8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:26.335785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=75 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:24:26.393191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd72465-bde7-405e-926e-5a86ba0c8f26 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:26.396928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53988b-eae5-48f6-a64e-0c62036736da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:26.397587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53988b-eae5-48f6-a64e-0c62036736da service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:26.397866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53988b-eae5-48f6-a64e-0c62036736da service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:26.398144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53988b-eae5-48f6-a64e-0c62036736da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:26.409325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53988b-eae5-48f6-a64e-0c62036736da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:26.411015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53988b-eae5-48f6-a64e-0c62036736da service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:26.423124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 755/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:24:29.012036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78018f85-0eb3-4146-9b72-03f807c378d3 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:29.012638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78018f85-0eb3-4146-9b72-03f807c378d3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:29.012951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78018f85-0eb3-4146-9b72-03f807c378d3 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:29.014502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 782/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 4 09:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:24:29.021675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923a46-6b79-4847-b5db-0c407eb6ae11 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:29.022284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923a46-6b79-4847-b5db-0c407eb6ae11 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:29.022284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923a46-6b79-4847-b5db-0c407eb6ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:29.044509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5d923a46-6b79-4847-b5db-0c407eb6ae11 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:24:29.046699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5d923a46-6b79-4847-b5db-0c407eb6ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:29.060936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 756/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 4 09:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:24:29.464139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78018f85-0eb3-4146-9b72-03f807c378d3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:24:29.466005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-78018f85-0eb3-4146-9b72-03f807c378d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:24:29.479919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78018f85-0eb3-4146-9b72-03f807c378d3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:24:29.484040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b301f35-0637-495d-9dba-1fca8bd9142c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:24:29.484755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b301f35-0637-495d-9dba-1fca8bd9142c service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:24:29.485047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b301f35-0637-495d-9dba-1fca8bd9142c service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:24:29.485330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b301f35-0637-495d-9dba-1fca8bd9142c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:24:29.496609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b301f35-0637-495d-9dba-1fca8bd9142c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:24:29.498140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b301f35-0637-495d-9dba-1fca8bd9142c service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:24:29.509742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 783/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:24:39.035405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=28 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:24:39.504763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=8 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:03.762224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fe123-00db-4b3f-8dca-1293c1f4c61d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.763225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fe123-00db-4b3f-8dca-1293c1f4c61d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.763513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fe123-00db-4b3f-8dca-1293c1f4c61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.798242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-2a2fe123-00db-4b3f-8dca-1293c1f4c61d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:03.800903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2a2fe123-00db-4b3f-8dca-1293c1f4c61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:03.815142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 757/1540] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:03.825285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7499b52-dbb6-45d6-882f-4253321db205 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.826008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7499b52-dbb6-45d6-882f-4253321db205 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.826105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7499b52-dbb6-45d6-882f-4253321db205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.828207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.828855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.829375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.858591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:03.860621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:03.861510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-f7499b52-dbb6-45d6-882f-4253321db205 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:03.863231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f7499b52-dbb6-45d6-882f-4253321db205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:03.879783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 784/1542] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:03.880739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 758/1542] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:03.880971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6f897-c050-4cda-a72a-dcdaf114ca4f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.881804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6f897-c050-4cda-a72a-dcdaf114ca4f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.881804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6f897-c050-4cda-a72a-dcdaf114ca4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.887674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:03.904209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a46f5d-38ff-4418-a69c-90d6bf7cd348 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:03.915594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:03.916953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.917488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-47c6f897-c050-4cda-a72a-dcdaf114ca4f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:03.917801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.918307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.919282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-47c6f897-c050-4cda-a72a-dcdaf114ca4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:03.919722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:03.920261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:03.922527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeec7e8-ad3f-471a-aaeb-30757c3e24ee admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:03.939476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 785/1543] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:03.941393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.941723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.942020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.951706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 759/1544] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:26:03.953702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:03.954608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:03.955146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:03.982558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:03.982558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:03.993405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:03.995519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:04.006829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 786/1545] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.008392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.017949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 760/1546] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.019340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.028079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08d939-049d-407e-99d8-dba2d8d66adf None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.038214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581761-87d8-4604-8ad5-6f63a5084490 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.040635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.041386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.041633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.041915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.042641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.044794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.057939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.059287 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.059810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.060377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.061374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:04.061932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.063834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.093775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 761/1547] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:04.095171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.113115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0963a2c-30ec-4860-8d34-f129374ecfcc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.126283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.127416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.127971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.128806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.129929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:04.130492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.132746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.154610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 762/1548] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:04.156153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.175693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ff68b-02b6-4c28-b5cc-a1b32f30dc55 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.185367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.186514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.187041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.187533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.188511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.190218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:26:04.190218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:26:04.190218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:26:04.191934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef8317-751c-4c14-81c0-ded6e4eb21ad admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.307517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 787/1549] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/projects => generated 357 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:04.308815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.356141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 763/1550] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:04.358830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.359447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.359747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.412702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:04.414391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:04.439333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672529e-ae07-42cf-a390-91266f4c6b1c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.534278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f9a81-9dcd-4e8a-9411-c53e386e0a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.535846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f9a81-9dcd-4e8a-9411-c53e386e0a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.536136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f9a81-9dcd-4e8a-9411-c53e386e0a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.536444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f9a81-9dcd-4e8a-9411-c53e386e0a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.538284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f9a81-9dcd-4e8a-9411-c53e386e0a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.543131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f9a81-9dcd-4e8a-9411-c53e386e0a7c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.564921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 764/1551] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.569846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.579068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559da57-a60e-4837-a6cc-68ac6d570a22 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.584271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.585072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.585358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.585657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.586423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.588705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.748811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 765/1552] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/users => generated 279 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:04.749862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.762282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f06bb8-8355-4a14-bda8-c053a302dc8c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.768118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49faf9d9-0a99-473e-9c39-d35ef28bbfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.769038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49faf9d9-0a99-473e-9c39-d35ef28bbfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.769249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49faf9d9-0a99-473e-9c39-d35ef28bbfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.769481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49faf9d9-0a99-473e-9c39-d35ef28bbfc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.770231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49faf9d9-0a99-473e-9c39-d35ef28bbfc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.772199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49faf9d9-0a99-473e-9c39-d35ef28bbfc3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.775807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 788/1553] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/projects => generated 393 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:04.778984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5084-7fca-4cac-b30b-900ade59528c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.778984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5084-7fca-4cac-b30b-900ade59528c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.778984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5084-7fca-4cac-b30b-900ade59528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.801014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-26be5084-7fca-4cac-b30b-900ade59528c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:04.802156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-26be5084-7fca-4cac-b30b-900ade59528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:04.859829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:04.877480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 789/1554] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.884359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442593c-68db-48e0-a309-9b27b479ca67 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.884359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442593c-68db-48e0-a309-9b27b479ca67 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.884359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442593c-68db-48e0-a309-9b27b479ca67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.901983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9442593c-68db-48e0-a309-9b27b479ca67 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:04.903407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9442593c-68db-48e0-a309-9b27b479ca67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:04.917200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 766/1555] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:04.936540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 790/1556] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.936763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.936763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.936763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.937195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.937195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.937195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.950064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:04.950884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:04.952094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:04.952846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:04.971676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 767/1557] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.971999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 791/1558] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:04.972177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.972404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:04.983118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc0a87-92ad-4615-bc6a-b0ec88db0e97 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.988320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901f209-37e0-4282-a190-9aeb3963466a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:04.989009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.989872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.990232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.990601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.991658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.993062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:26:04.993062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:26:04.993062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:26:04.994800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:04.995032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea72ba-114b-4716-a782-e7caf43cb7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:04.995820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea72ba-114b-4716-a782-e7caf43cb7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:04.996193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea72ba-114b-4716-a782-e7caf43cb7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:04.996604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea72ba-114b-4716-a782-e7caf43cb7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:04.997553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea72ba-114b-4716-a782-e7caf43cb7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:04.999427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea72ba-114b-4716-a782-e7caf43cb7d1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.014321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 768/1559] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:05.016683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d92a-ff26-42d7-b767-092ba5fda2dc None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.017184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d92a-ff26-42d7-b767-092ba5fda2dc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.017898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d92a-ff26-42d7-b767-092ba5fda2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.039541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c6a2d92a-ff26-42d7-b767-092ba5fda2dc None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:05.040907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c6a2d92a-ff26-42d7-b767-092ba5fda2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:05.049469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 769/1560] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:05.050852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.051195 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.051490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.071520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:05.072831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:05.081434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 770/1561] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:05.082326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.093216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8579343-cc3b-481e-95dd-41e12bb90f90 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.098240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d080b-19aa-457f-8a22-89ba2040e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.098933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d080b-19aa-457f-8a22-89ba2040e616 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.099216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d080b-19aa-457f-8a22-89ba2040e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.099497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d080b-19aa-457f-8a22-89ba2040e616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.100237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d080b-19aa-457f-8a22-89ba2040e616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.101946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d080b-19aa-457f-8a22-89ba2040e616 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.121260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 771/1562] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:05.121260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110330-5038-4f1f-8e41-b5f83fbe9d37 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.121675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110330-5038-4f1f-8e41-b5f83fbe9d37 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.121911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110330-5038-4f1f-8e41-b5f83fbe9d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.134165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 792/1563] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:04 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:05.134165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.140208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:05.147070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320631b8-1406-4f8e-966c-f0df8d2575c9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.151366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-cf110330-5038-4f1f-8e41-b5f83fbe9d37 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:05.152791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-cf110330-5038-4f1f-8e41-b5f83fbe9d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:05.153506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.154297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.154722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.155150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.156247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:05.156719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.158251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:26:05.158251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:26:05.158251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:26:05.159363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.162262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 772/1564] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:05.163947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81230f-0eb8-4dae-a142-c501926818e0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.164239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81230f-0eb8-4dae-a142-c501926818e0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.164602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81230f-0eb8-4dae-a142-c501926818e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.183099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 793/1565] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:05.184174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.186330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-cc81230f-0eb8-4dae-a142-c501926818e0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:05.188076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-cc81230f-0eb8-4dae-a142-c501926818e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:05.195768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555e3de-9aba-4b96-bbbf-fe327dbc12d7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.199682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 773/1566] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:05.202916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.202916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.202916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.203533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.204727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.205123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.205607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.206927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.210044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.226080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 794/1567] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:05.227272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.231123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:05.234207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:05.242026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39135809-da6e-4e9c-971f-2c9dcbb7967f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.246582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 774/1568] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:05.248022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.248237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.248968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.249261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.249547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.250300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:05.250614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.252415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.262092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98045c-085a-45e3-bd9f-0bf6271e86aa None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.270304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.272044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 795/1569] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:05.272713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.273000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.273425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.274095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.275636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.278964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.291495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3f2c-fc2a-48f0-b56e-f176f5d65f96 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.297180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.297972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.298334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.298689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.299156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 775/1570] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:05.299635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.300813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.301580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.315799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 796/1571] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:05.317658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.317857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c1639-8797-430c-837e-36a8da61ee14 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.325773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.326742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.327190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.327588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.328985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:05.329469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.331367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.336715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31b616-4a45-4115-bd46-14498d238e96 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.344904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.346053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.346468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.346945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.348267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.348655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 776/1572] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:05.349861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.352163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.369434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e78c2-1a67-43fb-90e5-e7564bc90ab7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.377484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.378467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.378930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.379343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.380799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:05.381361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.385016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.407717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 777/1573] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:05.409177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.427284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074bb0d-0429-45e1-b423-961822ab7077 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.435680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.437316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.437838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.438283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] PATH_INFO: `/v3/projects/e92b4e1df142437f9a01f58f7e638aa0/users/afc2a5334f7046769f70b9598f589407/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.466793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92b4e1df142437f9a01f58f7e638aa0, user_id=afc2a5334f7046769f70b9598f589407, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.468689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:26:05.468689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:26:05.468689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:26:05.477981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.478110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 797/1574] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/projects => generated 393 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:05.478110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.552458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0cb2-f528-41dc-b566-743a660f2975 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.557529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.563914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.563914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.563914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.563914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:05.563914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.563914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.575968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 798/1575] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:05.576936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.588094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60972912-93bc-4b93-b975-a2d281aa2a32 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.593757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.594868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.595243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.595659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.596576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.599095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.605240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 778/1576] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:05 2022] PUT /identity/v3/projects/e92b4e1df142437f9a01f58f7e638aa0/users/afc2a5334f7046769f70b9598f589407/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:05.624654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.697512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd939-4e4e-4ace-887a-43fa7b36d5f7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.702452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.703377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.703657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.703932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] PATH_INFO: `/v3/projects/e27e8df485ff405e965cf22847c3b262/users/ac869ff528aa49989c88b3a545ba9cf1/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.721480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27e8df485ff405e965cf22847c3b262, user_id=ac869ff528aa49989c88b3a545ba9cf1, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.724469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.783400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 799/1577] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:05.783400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.813849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:05.822259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 779/1578] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:05 2022] PUT /identity/v3/projects/e27e8df485ff405e965cf22847c3b262/users/ac869ff528aa49989c88b3a545ba9cf1/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:05.824063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.865132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3899c7-93c6-4d9b-a71f-3000d71da7d9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.908689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a847f-b7a1-482c-a11a-f7baa1b1381d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.914037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.915475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.915763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.916049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.916870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.918819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.928107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:05.948441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.949149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.949433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.949712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.950457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.952115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:05.964141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 800/1579] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:05.965488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:05.978282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e28c-3c03-4680-89ed-75c5a2efdfe6 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:05.983384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:05.984068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:05.984417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:05.984693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:05.985408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:05.987402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.121405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 780/1580] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/projects => generated 379 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.122198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.133254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 801/1581] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:05 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.134219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.234358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d790828-4a85-4257-ac34-b12d5d10e62d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.240016 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53ee23-6011-47b7-a9a3-1a9d5205ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.240665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9487af-d962-4513-8dc2-3ab66d24febc admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.240973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53ee23-6011-47b7-a9a3-1a9d5205ca75 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.241347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53ee23-6011-47b7-a9a3-1a9d5205ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.241794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53ee23-6011-47b7-a9a3-1a9d5205ca75 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.243190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53ee23-6011-47b7-a9a3-1a9d5205ca75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.246215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53ee23-6011-47b7-a9a3-1a9d5205ca75 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.249124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.250311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.250815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.251320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.252664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.256450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.401121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 781/1582] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.405963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0e118-a2ce-49cd-bcf6-d209257265f0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.405963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0e118-a2ce-49cd-bcf6-d209257265f0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.405963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0e118-a2ce-49cd-bcf6-d209257265f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.439378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dcf0e118-a2ce-49cd-bcf6-d209257265f0 None None] MFA Rules not processed for user `afc2a5334f7046769f70b9598f589407`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:06.441089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dcf0e118-a2ce-49cd-bcf6-d209257265f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:06.446015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 802/1583] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/projects => generated 387 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.447138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.489546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:06.537490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 782/1584] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:06.540063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.540652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.541158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.551617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8a0f-3912-4f59-85b9-defbfdb5a2d0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.559519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.561025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.561547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.562071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.563409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.567179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.572142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:06.574352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:06.587793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 783/1585] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:06.589243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.601133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b71752-9a7e-44ed-9030-a5a56f16acf3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.606196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.607143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.607406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.607905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] PATH_INFO: `/v3/projects/e42983d2d3964b8cb5d32e6d653d6dfd/users/34a7ef6d33b141ce8fb1cec517faffd1/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.628088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e42983d2d3964b8cb5d32e6d653d6dfd, user_id=34a7ef6d33b141ce8fb1cec517faffd1, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.632165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.729856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 803/1586] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.731095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.738252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 784/1587] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:06 2022] PUT /identity/v3/projects/e42983d2d3964b8cb5d32e6d653d6dfd/users/34a7ef6d33b141ce8fb1cec517faffd1/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:06.739288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.747941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72c819-0944-4cef-88f3-055505bed837 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.813474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:06.825749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.826716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.826885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.827094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.827903 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.829983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.840877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95a28-a52f-4e23-bba3-e42d19df5479 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.847152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.849798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.849798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.849798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.850384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.852826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.953751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 804/1588] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.954795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.965775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879462ec-b6ca-4e1b-bef3-09177c59c67d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.971346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:06.972065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:06.972403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:06.972729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:06.973773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:06.975506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:06.978575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 785/1589] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:06.979963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.991967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555e34c-3ca9-4de7-8f1b-b594db2a4a49 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:06.996498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 805/1590] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:06.997951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:06.999408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.000458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.000894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.001294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.002130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.004255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.015563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f82c2a-6811-4a3b-8232-6fd1b56977d8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.020784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.021497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.021775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.022052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.022756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.024749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.099390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 786/1591] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:07.101091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.140645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 806/1592] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:06 2022] POST /identity/v3/projects => generated 355 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:07.142261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.169651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a819e5-d9ce-4078-8e58-25a9919d1baf admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.231227 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.231825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.232113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.232437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.251229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965e5ca-33c2-4cc0-970e-7b558933ede6 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.261422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618b09-71de-4e63-a412-f516947ab74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.262866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618b09-71de-4e63-a412-f516947ab74f admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.263381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618b09-71de-4e63-a412-f516947ab74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.263883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618b09-71de-4e63-a412-f516947ab74f admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.265497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3618b09-71de-4e63-a412-f516947ab74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.268739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3618b09-71de-4e63-a412-f516947ab74f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.285724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 807/1593] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.285724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.285724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.285724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.319216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:07.321535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:07.330301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.333180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.335540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 808/1594] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:07.337117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.348059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:07.352539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 787/1595] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:07.354008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.356092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef310c8-176d-47df-b681-e48b8f33048a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.364977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.371083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.371300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b289-4d1a-41ca-8227-518779ae7797 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.371491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.371491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.371491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.372784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.374538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.375368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.375758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.376142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.377050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.379014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.389037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 809/1596] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.390098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.396658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 788/1597] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.396658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.401594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd6ca-7d1f-4293-aa14-28c20172bd84 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.405280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:07.407427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.408188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.408504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.408783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.409525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.411388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.411795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a57df-3f15-4a66-a21b-88fdb13e5e7d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.418702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.419398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.419680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.419990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.420979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.422622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.424423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 810/1598] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.425305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.434373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 789/1599] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.435276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.443847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f9fe7-ea32-4401-97b0-42f603a3699a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.447589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81eb945-daea-4da3-a060-5f8bdec5c106 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.450295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.451553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.452111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.452703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.452854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.453445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.453729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.454007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.454145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.454746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.456638 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.458235 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.470247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 790/1600] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.471573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.484027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683285-c1db-4fb9-909d-974d28e6439a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.488602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.489797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.490378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.490871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.509148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.512408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.534057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 791/1601] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:07.536106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.554175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf5803-7975-4e7c-8331-101fd3a8337c service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.563982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.565185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.565706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.566276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.567640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.570954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.588775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 792/1602] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.590645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.608873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ac6e8-d8f5-4bf6-b93f-931833355b68 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.617799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.618968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.619468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.619979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.621469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.624648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.637261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 811/1603] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:07 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:07.641753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.651536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 793/1604] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.651536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.662011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae5c2f-f017-4468-a6c5-7e7e9111b18f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.667222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e952a5b-daf3-41d8-ac89-fbe1604aa7e1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.674972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.675653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.676659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.676851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.677192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.677337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.677701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] PATH_INFO: `/v3/projects/2b7fc9192e874eb9bed73cfceace70b4/users/f3e8dcdf9b9d4900b60224e94426c8c0/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.677801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.679232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.682911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.707364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 794/1605] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:07.707364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.715381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7fc9192e874eb9bed73cfceace70b4, user_id=f3e8dcdf9b9d4900b60224e94426c8c0, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.717963 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.725826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3080d-b212-4458-97b6-13acf317a92c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:07.734572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:07.736019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:07.736575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:07.737073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] PATH_INFO: `/v3/projects/64e9b1acaed94e648a8c2debe67bee7c/users/2059259463264cab9e1b7da51a61a6cc/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:07.758997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e9b1acaed94e648a8c2debe67bee7c, user_id=2059259463264cab9e1b7da51a61a6cc, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:07.762367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:07.859031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 812/1606] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:07 2022] PUT /identity/v3/projects/2b7fc9192e874eb9bed73cfceace70b4/users/f3e8dcdf9b9d4900b60224e94426c8c0/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:07.860739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.919767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 795/1607] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:07 2022] PUT /identity/v3/projects/64e9b1acaed94e648a8c2debe67bee7c/users/2059259463264cab9e1b7da51a61a6cc/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:07.920910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:07.999134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcf525-7360-4fef-b9ff-e5bcab279287 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:08.006865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacd692-ec48-4cd7-99b7-fe97913f6c9a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:08.012530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:08.012530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.012530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.012530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] PATH_INFO: `/v3/projects/29bb9979415948f685ad1fe8d58bfb20/users/c1a0478618b1430b9c8c609618257eca/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.013110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:08.013959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.014348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.014727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.016443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:08.019397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:08.033348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 796/1608] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:08.033348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:08.039692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29bb9979415948f685ad1fe8d58bfb20, user_id=c1a0478618b1430b9c8c609618257eca, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:08.045337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:08.052745 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca893-b21b-400c-b4e1-22c749fd5d82 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:08.053840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e720-973b-4fee-b26f-8c9cb5f94fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:08.055485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e720-973b-4fee-b26f-8c9cb5f94fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.055769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e720-973b-4fee-b26f-8c9cb5f94fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.056041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e720-973b-4fee-b26f-8c9cb5f94fd2 admin admin] PATH_INFO: `/v3/projects/7410684ee8db448d8385de8549bce1a7/users/aeffa44fe38a4204ae21b52fe827cedc/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.074101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e720-973b-4fee-b26f-8c9cb5f94fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7410684ee8db448d8385de8549bce1a7, user_id=aeffa44fe38a4204ae21b52fe827cedc, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:08.077027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e720-973b-4fee-b26f-8c9cb5f94fd2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:08.176036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 813/1609] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:07 2022] PUT /identity/v3/projects/29bb9979415948f685ad1fe8d58bfb20/users/c1a0478618b1430b9c8c609618257eca/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:08.181154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:08.259420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 797/1610] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:08 2022] PUT /identity/v3/projects/7410684ee8db448d8385de8549bce1a7/users/aeffa44fe38a4204ae21b52fe827cedc/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:08.259420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ba1fc-c1ad-47b2-8915-22a4522e29a8 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.259420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ba1fc-c1ad-47b2-8915-22a4522e29a8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.259420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ba1fc-c1ad-47b2-8915-22a4522e29a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.279275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a5d57-e65a-4284-b970-04848c2a1788 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:08.297916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-991ba1fc-c1ad-47b2-8915-22a4522e29a8 None None] MFA Rules not processed for user `f3e8dcdf9b9d4900b60224e94426c8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:08.299523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-991ba1fc-c1ad-47b2-8915-22a4522e29a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:08.359412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:08.375367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:08.439760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d319df2-3409-45c1-b73e-b434813aa227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:08.441997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d319df2-3409-45c1-b73e-b434813aa227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.442614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d319df2-3409-45c1-b73e-b434813aa227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.443181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d319df2-3409-45c1-b73e-b434813aa227 admin admin] PATH_INFO: `/v3/projects/0cf07261800e47938626f61a1fb73b2b/users/7b6676edfa984008b846589b2df9ef92/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.449552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 798/1611] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:08.451955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.452879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.453354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.488106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d319df2-3409-45c1-b73e-b434813aa227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf07261800e47938626f61a1fb73b2b, user_id=7b6676edfa984008b846589b2df9ef92, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:08.492915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] MFA Rules not processed for user `2059259463264cab9e1b7da51a61a6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:08.494170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:08.494509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d319df2-3409-45c1-b73e-b434813aa227 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:08.597392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 799/1612] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:08.597392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:08.623443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190d6d2-9286-4465-8576-cb15e54e6c5b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:08.632986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8c911-08c9-47bb-b033-b14161594a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:08.634605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8c911-08c9-47bb-b033-b14161594a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.635111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8c911-08c9-47bb-b033-b14161594a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.635604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8c911-08c9-47bb-b033-b14161594a53 admin admin] PATH_INFO: `/v3/projects/23f464f9a78e4698a2002c96c48867cb/users/a51b41b3119248f39ff2c32d8e532b4e/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.655904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 814/1613] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:08 2022] PUT /identity/v3/projects/0cf07261800e47938626f61a1fb73b2b/users/7b6676edfa984008b846589b2df9ef92/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:08.657434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cb13a-bea1-4bfb-b63e-05dda937feeb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.657831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cb13a-bea1-4bfb-b63e-05dda937feeb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.658211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cb13a-bea1-4bfb-b63e-05dda937feeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.663086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8c911-08c9-47bb-b033-b14161594a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f464f9a78e4698a2002c96c48867cb, user_id=a51b41b3119248f39ff2c32d8e532b4e, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:08.666934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8c911-08c9-47bb-b033-b14161594a53 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:08.677516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2f9cb13a-bea1-4bfb-b63e-05dda937feeb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:08.678773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2f9cb13a-bea1-4bfb-b63e-05dda937feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:08.743880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 815/1614] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:08.745076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.745368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.745646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.767865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:08.769150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:08.785068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 816/1615] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:08.785068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:08.795097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef482-924a-45b3-8473-cc14e51da918 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:08.801953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 800/1616] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:08 2022] PUT /identity/v3/projects/23f464f9a78e4698a2002c96c48867cb/users/a51b41b3119248f39ff2c32d8e532b4e/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:08.805869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd19d-f641-4ad2-bf4b-b7c7e04dc6a9 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.806617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd19d-f641-4ad2-bf4b-b7c7e04dc6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.807127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd19d-f641-4ad2-bf4b-b7c7e04dc6a9 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.813651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 801/1617] 10.222.0.46 () {54 vars in 961 bytes} [Sat Jun 4 09:26:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:08.816450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.816973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.817491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.857512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:08.860055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:08.885077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:08.886306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:08.886669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:08.887279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:08.888150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:08.890761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:08.970089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 802/1618] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:08.971306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.037926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:09.060685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9205050-9559-445d-8aa8-0da7271900ee None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.067341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.073913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.073913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.073913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.074414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 817/1619] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:08 2022] POST /identity/v3/projects => generated 391 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:09.074414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.094831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.100231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.120230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 803/1620] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:09.122146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.138783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:09.140365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b70c5-d3fa-4da4-a15e-415b251d4252 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.202466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58a287-d9f3-4234-9ab8-9ac50d22cb42 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.214324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5b2e-9e6a-4d84-b137-7132f0bec813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.215690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5b2e-9e6a-4d84-b137-7132f0bec813 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.216199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5b2e-9e6a-4d84-b137-7132f0bec813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.216797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5b2e-9e6a-4d84-b137-7132f0bec813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.218810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5b2e-9e6a-4d84-b137-7132f0bec813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.222593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5b2e-9e6a-4d84-b137-7132f0bec813 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.230297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.231362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.232919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.233404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.247684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.249595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.253765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 818/1621] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:09.258134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.258660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.260146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.271657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 804/1622] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:09.272784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.285442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef18de-698a-427b-a651-722a44d7ae36 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.300768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.303131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.303604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:09.303723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.305009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.306576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:09.333303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.337367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.362698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 805/1623] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:09.363920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.379023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca8cc9-c730-4d82-9e31-987bf14b9be5 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.386407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034048b-7c01-49dc-bfb5-8be6edced6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.387781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034048b-7c01-49dc-bfb5-8be6edced6bb admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.388086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034048b-7c01-49dc-bfb5-8be6edced6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.388463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034048b-7c01-49dc-bfb5-8be6edced6bb admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.389232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034048b-7c01-49dc-bfb5-8be6edced6bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.391910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034048b-7c01-49dc-bfb5-8be6edced6bb admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.410268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 819/1624] 10.222.0.46 () {58 vars in 1058 bytes} [Sat Jun 4 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:09.422104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.431060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577582-9596-48fa-8fd3-dd0ad879874e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.436347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.437257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.437538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.437814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] PATH_INFO: `/v3/projects/23f464f9a78e4698a2002c96c48867cb/users/a51b41b3119248f39ff2c32d8e532b4e/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.440757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f464f9a78e4698a2002c96c48867cb, user_id=a51b41b3119248f39ff2c32d8e532b4e, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.443610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.457805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 820/1625] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:09 2022] PUT /identity/v3/projects/23f464f9a78e4698a2002c96c48867cb/users/a51b41b3119248f39ff2c32d8e532b4e/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:09.467937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.541018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 806/1626] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:09 2022] POST /identity/v3/users => generated 302 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:09.543357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.543357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.543357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.575796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0948d35-9410-44d1-8e0b-dbe334cfa714 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.581142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] MFA Rules not processed for user `a51b41b3119248f39ff2c32d8e532b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:09.582398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:09.588098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.589741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.590234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.591092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.733732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 807/1627] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:09.734264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.734416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.734580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.762864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 821/1628] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:09.764479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:09.903496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a6a0-7112-4629-ba95-62965cd903dc service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.909782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.909782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.909782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.909782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.923084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.925636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:09.938691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 822/1629] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:09.968305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710bcd-3dc8-4aee-bd45-5f1d30b79d4d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:09.984200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:09.985313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:09.985578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:09.989597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:09.990640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:09.995445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:10.028980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:10.036872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 808/1630] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:10.054259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8b389-1812-4a09-81b8-bee4abf0d0d5 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:10.058112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:10.058602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.058995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.059376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] PATH_INFO: `/v3/projects/da46d50caf4d4767a4fe522d85bbe513/users/834d4059836042c0a78e31a16d6590a5/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.079403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da46d50caf4d4767a4fe522d85bbe513, user_id=834d4059836042c0a78e31a16d6590a5, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:10.082832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:10.253056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 823/1631] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:10 2022] PUT /identity/v3/projects/da46d50caf4d4767a4fe522d85bbe513/users/834d4059836042c0a78e31a16d6590a5/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:10.258576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:10.285944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9eac3c-db96-42fc-9d0d-aeb5d94475bc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:10.470219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:10.472575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:10.512878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164babbd-091f-4823-b845-6ca5031597a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:10.514031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164babbd-091f-4823-b845-6ca5031597a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.514499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164babbd-091f-4823-b845-6ca5031597a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.516029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164babbd-091f-4823-b845-6ca5031597a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.523406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164babbd-091f-4823-b845-6ca5031597a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:10.523406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164babbd-091f-4823-b845-6ca5031597a2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:10.552588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 809/1632] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:10.556958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c086a-37ac-4e90-aaaa-39c624d15730 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.557572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c086a-37ac-4e90-aaaa-39c624d15730 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.558080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c086a-37ac-4e90-aaaa-39c624d15730 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.561245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 810/1633] 10.222.0.46 () {54 vars in 912 bytes} [Sat Jun 4 09:26:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:10.563967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c086a-37ac-4e90-aaaa-39c624d15730 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:10.589389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:10.592853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c086a-37ac-4e90-aaaa-39c624d15730 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:10.602438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc9b10-faf8-450c-8cc8-b2bcd105c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:10.604598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9b10-faf8-450c-8cc8-b2bcd105c826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.605075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9b10-faf8-450c-8cc8-b2bcd105c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.606421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9b10-faf8-450c-8cc8-b2bcd105c826 admin admin] PATH_INFO: `/v3/projects/da46d50caf4d4767a4fe522d85bbe513/users/834d4059836042c0a78e31a16d6590a5/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.612909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc9b10-faf8-450c-8cc8-b2bcd105c826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da46d50caf4d4767a4fe522d85bbe513, user_id=834d4059836042c0a78e31a16d6590a5, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:10.618977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc9b10-faf8-450c-8cc8-b2bcd105c826 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:10.644048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd901-91c1-4124-8838-c8f8eac8823b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:10.650365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 811/1634] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:10 2022] PUT /identity/v3/projects/da46d50caf4d4767a4fe522d85bbe513/users/834d4059836042c0a78e31a16d6590a5/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:10.656218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42797c-12e0-4e68-bac0-2b16afaa8c96 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.656937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42797c-12e0-4e68-bac0-2b16afaa8c96 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.657711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42797c-12e0-4e68-bac0-2b16afaa8c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.706023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-6d42797c-12e0-4e68-bac0-2b16afaa8c96 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:10.709848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6d42797c-12e0-4e68-bac0-2b16afaa8c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:10.839682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:10.841843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.842678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.843431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.906367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 812/1635] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:26:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:10.909585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:10.910058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:10.910918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:10.998266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] MFA Rules not processed for user `834d4059836042c0a78e31a16d6590a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:10.998266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:11.019527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:11.021201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:11.063284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 824/1636] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:10 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:11.065353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:11.098218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1a35-3e86-42bf-a0ce-90a4538e6802 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:11.120744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:11.121919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:11.122433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:11.122943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:11.175179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:11.184553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:11.206826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 813/1637] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:11.208684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 825/1638] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:12.208005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:12.231064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56fb5d-7092-4e85-b4f7-3bae48eb9cee None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:12.246384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c9d4a-dee7-4045-a808-10d67f608608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:12.248268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c9d4a-dee7-4045-a808-10d67f608608 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:12.248606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c9d4a-dee7-4045-a808-10d67f608608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:12.249230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c9d4a-dee7-4045-a808-10d67f608608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:12.273374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c9d4a-dee7-4045-a808-10d67f608608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:12.275617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c9d4a-dee7-4045-a808-10d67f608608 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:12.297907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 814/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:12.486531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:12.499742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a12ec-d4d5-4185-8c6f-f8a5559c3cc4 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:12.508055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb49248-9cb5-42e5-a6a3-d4e0d225a073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:12.509261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49248-9cb5-42e5-a6a3-d4e0d225a073 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:12.509694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49248-9cb5-42e5-a6a3-d4e0d225a073 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:12.510319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49248-9cb5-42e5-a6a3-d4e0d225a073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:12.652359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:12.659999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb49248-9cb5-42e5-a6a3-d4e0d225a073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:12.664911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb49248-9cb5-42e5-a6a3-d4e0d225a073 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:12.700695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 826/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:12.910504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:13.167921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:13.167921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:13.167921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:13.216516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] MFA Rules not processed for user `c1a0478618b1430b9c8c609618257eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:13.218031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:13.421018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 815/1641] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:13.434562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efbff-158d-4911-99c1-9f3991f924e5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:13.435863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efbff-158d-4911-99c1-9f3991f924e5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:13.436395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efbff-158d-4911-99c1-9f3991f924e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:13.489084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bf0efbff-158d-4911-99c1-9f3991f924e5 None None] MFA Rules not processed for user `c1a0478618b1430b9c8c609618257eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:13.491781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bf0efbff-158d-4911-99c1-9f3991f924e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:13.517461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 827/1642] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:13.529696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:13.552639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7adb-1e20-4bce-9449-87d31caa1a94 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:13.576378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce86cb-d950-44c0-a3d9-ff2cd3f0b5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:13.577563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce86cb-d950-44c0-a3d9-ff2cd3f0b5cb service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:13.589999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce86cb-d950-44c0-a3d9-ff2cd3f0b5cb service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:13.590463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce86cb-d950-44c0-a3d9-ff2cd3f0b5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:13.629829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce86cb-d950-44c0-a3d9-ff2cd3f0b5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:13.632356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce86cb-d950-44c0-a3d9-ff2cd3f0b5cb service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:13.668738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 816/1643] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:13.925061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a063b6-5161-4e03-9687-0445eb78208b None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:13.925604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a063b6-5161-4e03-9687-0445eb78208b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:13.926035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a063b6-5161-4e03-9687-0445eb78208b None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:13.932249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 828/1644] 10.222.0.46 () {54 vars in 961 bytes} [Sat Jun 4 09:26:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:13.944787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296568e-80d8-4dbb-8d51-fb811830d908 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:13.945326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296568e-80d8-4dbb-8d51-fb811830d908 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:13.945792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296568e-80d8-4dbb-8d51-fb811830d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:13.970169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f296568e-80d8-4dbb-8d51-fb811830d908 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:13.971358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f296568e-80d8-4dbb-8d51-fb811830d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:14.006698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 817/1645] 10.222.0.46 () {58 vars in 1058 bytes} [Sat Jun 4 09:26:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:14.018676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dfdf4-900c-401b-bc35-5bdf54082f09 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:14.025831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dfdf4-900c-401b-bc35-5bdf54082f09 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:14.025831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dfdf4-900c-401b-bc35-5bdf54082f09 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:14.031078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 829/1646] 10.222.0.46 () {54 vars in 912 bytes} [Sat Jun 4 09:26:14 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:14.055236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e082b-959f-442c-bf11-a7475e649a86 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:14.056095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e082b-959f-442c-bf11-a7475e649a86 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:14.056650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e082b-959f-442c-bf11-a7475e649a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:14.090194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5e2e082b-959f-442c-bf11-a7475e649a86 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:14.091823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5e2e082b-959f-442c-bf11-a7475e649a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:14.131426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 818/1647] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:26:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:14.189954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dfdf4-900c-401b-bc35-5bdf54082f09 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:14.204752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dfdf4-900c-401b-bc35-5bdf54082f09 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:14.216170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f1e80-7df9-4b75-aee5-ef04da972e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:14.216994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f1e80-7df9-4b75-aee5-ef04da972e96 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:14.217315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f1e80-7df9-4b75-aee5-ef04da972e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:14.217600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f1e80-7df9-4b75-aee5-ef04da972e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:14.249061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f1e80-7df9-4b75-aee5-ef04da972e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:14.252716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f1e80-7df9-4b75-aee5-ef04da972e96 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:14.304608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 830/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:14.542680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b65dc-ec68-455d-8369-20392524e483 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:14.543155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b65dc-ec68-455d-8369-20392524e483 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:14.543222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b65dc-ec68-455d-8369-20392524e483 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:14.545206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 819/1649] 10.222.0.46 () {54 vars in 912 bytes} [Sat Jun 4 09:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:14.571483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ae93-f4cb-4dcc-876b-05fa6b1b84f1 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:14.571990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ae93-f4cb-4dcc-876b-05fa6b1b84f1 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:14.572189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ae93-f4cb-4dcc-876b-05fa6b1b84f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:14.625143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9988ae93-f4cb-4dcc-876b-05fa6b1b84f1 None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:14.627664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9988ae93-f4cb-4dcc-876b-05fa6b1b84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:14.707884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 831/1650] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:26:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:14.727581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b65dc-ec68-455d-8369-20392524e483 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:14.738473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b65dc-ec68-455d-8369-20392524e483 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:14.743052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b6baf-f41c-4cb8-bf5d-4b8e24f13120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:14.743829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6baf-f41c-4cb8-bf5d-4b8e24f13120 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:14.744186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6baf-f41c-4cb8-bf5d-4b8e24f13120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:14.744572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6baf-f41c-4cb8-bf5d-4b8e24f13120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:14.754500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b6baf-f41c-4cb8-bf5d-4b8e24f13120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:14.757488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b6baf-f41c-4cb8-bf5d-4b8e24f13120 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:14.770436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 820/1651] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:15.690723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9def7b-485b-4043-a384-d35235f0ff81 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:15.691559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9def7b-485b-4043-a384-d35235f0ff81 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:15.691559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9def7b-485b-4043-a384-d35235f0ff81 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:15.692831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 832/1652] 10.222.0.46 () {54 vars in 960 bytes} [Sat Jun 4 09:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:26:15.698972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:15.699790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:15.699790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:15.736997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:15.736997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:15.867642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 821/1653] 10.222.0.46 () {58 vars in 1057 bytes} [Sat Jun 4 09:26:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:15.877062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9def7b-485b-4043-a384-d35235f0ff81 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:15.907952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9def7b-485b-4043-a384-d35235f0ff81 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:15.914796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:15.915716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:15.916148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:15.916611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:15.925973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:15.947309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:15.951319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34d15-3a66-45cf-95dc-11cccd7686f9 service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:15.966636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 833/1654] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:23.084920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:23.163423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:23.513749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd56819-9a00-41a0-9c60-ddf3d62e75f1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:23.540717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:23.543394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:23.544208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:23.544934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] PATH_INFO: `/v3/users/f3e8dcdf9b9d4900b60224e94426c8c0` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:23.547521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e8dcdf9b9d4900b60224e94426c8c0)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:23.549827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: WARNING py.warnings [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:26:23.549827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: warnings.warn(msg) Jun 04 09:26:23.549827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: Jun 04 09:26:23.553565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:24.053217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e8dcdf9b9d4900b60224e94426c8c0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:24.054770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e8dcdf9b9d4900b60224e94426c8c0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:24.113320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e8dcdf9b9d4900b60224e94426c8c0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:24.235441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e8dcdf9b9d4900b60224e94426c8c0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:24.236799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e8dcdf9b9d4900b60224e94426c8c0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:24.442933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 822/1655] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:23 2022] DELETE /identity/v3/users/f3e8dcdf9b9d4900b60224e94426c8c0 => generated 0 bytes in 1361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:24.449221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b9ee3-0cb9-4676-8425-c2076b77e96b None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:24.450130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b9ee3-0cb9-4676-8425-c2076b77e96b None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:24.450754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b9ee3-0cb9-4676-8425-c2076b77e96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:24.527803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-df8b9ee3-0cb9-4676-8425-c2076b77e96b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:24.529156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-df8b9ee3-0cb9-4676-8425-c2076b77e96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:24.567470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:24.690635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 834/1656] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:24.793160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:24.795081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:24.861543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:24.864156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:24.865291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:25.033869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:25.045513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:25.045992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323603c-e66e-453c-85a8-d955d8902a7c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:25.065598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:25.075485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:25.076840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:25.079270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:25.142470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:25.150270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:25.201266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 823/1657] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:25.276899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:25.278770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:25.310407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624675b-758f-4a5d-8d95-daacc51ad6e4 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:25.329546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:25.331620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 835/1658] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:26:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:25.332483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:25.332777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:25.333107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] PATH_INFO: `/v3/users/2059259463264cab9e1b7da51a61a6cc` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:25.337586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2059259463264cab9e1b7da51a61a6cc)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:25.346600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:25.659286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:25.914644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2059259463264cab9e1b7da51a61a6cc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:25.917675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2059259463264cab9e1b7da51a61a6cc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:25.953817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2059259463264cab9e1b7da51a61a6cc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:25.981562 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2059259463264cab9e1b7da51a61a6cc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:25.983254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2e11f605-8786-45f3-9f44-fca749a9e04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2059259463264cab9e1b7da51a61a6cc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:26.009871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:26.012586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:26.184144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 824/1659] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:25 2022] DELETE /identity/v3/users/2059259463264cab9e1b7da51a61a6cc => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:26.189868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:26.190999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:26.191970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:26.226739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acec77-b67d-461f-96e6-d321e228b634 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:26.279902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:26.285320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:26.483173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:26.484750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:26.485088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:26.485376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:26.493116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:26.522736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 825/1660] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:26.522995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:26.575750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:26.577404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:26.715450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:26.717326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:26.777081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 836/1661] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:26:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 04 09:26:26.788566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69381dce-0d31-4da9-a213-c509a8ec226c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:26.808724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d154b-e8f6-486d-a3aa-1640089d6496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:26.809172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d154b-e8f6-486d-a3aa-1640089d6496 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:26.809662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d154b-e8f6-486d-a3aa-1640089d6496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:26.810354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d154b-e8f6-486d-a3aa-1640089d6496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:26.852676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d154b-e8f6-486d-a3aa-1640089d6496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:26.859859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d154b-e8f6-486d-a3aa-1640089d6496 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:26.889772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 826/1662] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:28.142159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:28.147279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:28.224577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdd67e-52a4-4083-8dd0-8be27f8268ee service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:28.238362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:28.240774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:28.241355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:28.241871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] PATH_INFO: `/v3/projects/2b7fc9192e874eb9bed73cfceace70b4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:28.255819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7fc9192e874eb9bed73cfceace70b4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:28.262428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: WARNING py.warnings [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 04 09:26:28.262428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: warnings.warn(msg) Jun 04 09:26:28.262428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: Jun 04 09:26:28.273794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:28.749535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:28.750898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:28.751222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:28.860680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] MFA Rules not processed for user `ac869ff528aa49989c88b3a545ba9cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:28.877034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:29.039899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 827/1663] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:29.066776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:29.068422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:29.422495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7fc9192e874eb9bed73cfceace70b4', 'request_id': 'req-29d941c4-6e45-487f-bfa8-0952f4e15601'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:29.512259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7fc9192e874eb9bed73cfceace70b4', 'request_id': 'req-29d941c4-6e45-487f-bfa8-0952f4e15601'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:29.536420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171380b-5bc2-40fb-bb94-11af7be382f3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:29.652541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7fc9192e874eb9bed73cfceace70b4', 'request_id': 'req-29d941c4-6e45-487f-bfa8-0952f4e15601'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:29.687243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] The token cache is being invalidate because project 2b7fc9192e874eb9bed73cfceace70b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:29.687763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:29.690065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:29.698020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 837/1664] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:28 2022] DELETE /identity/v3/projects/2b7fc9192e874eb9bed73cfceace70b4 => generated 0 bytes in 1557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:29.719036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:29.723550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:29.907029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d941c4-6e45-487f-bfa8-0952f4e15601 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:29.915304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:29.919634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:29.920853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:29.921306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:29.921742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] PATH_INFO: `/v3/users/834d4059836042c0a78e31a16d6590a5` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:29.926464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834d4059836042c0a78e31a16d6590a5)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:29.927302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:29.934844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:29.938091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:29.942823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:29.962547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:30.202815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:30.205012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116214f-7d43-4722-9252-6b73954d18d0 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:30.221395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 828/1665] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:30.260638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:30.304116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834d4059836042c0a78e31a16d6590a5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:30.354487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834d4059836042c0a78e31a16d6590a5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:30.357770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834d4059836042c0a78e31a16d6590a5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:30.459699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834d4059836042c0a78e31a16d6590a5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:30.460670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e67592dd-4c37-432c-9608-0a56e169e499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834d4059836042c0a78e31a16d6590a5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:30.618845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 838/1666] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:29 2022] DELETE /identity/v3/users/834d4059836042c0a78e31a16d6590a5 => generated 0 bytes in 901 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:30.924121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23a71-ed96-4499-ba72-5e0eedade437 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:30.924730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23a71-ed96-4499-ba72-5e0eedade437 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:30.926969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23a71-ed96-4499-ba72-5e0eedade437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:31.021511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a6d23a71-ed96-4499-ba72-5e0eedade437 None None] MFA Rules not processed for user `34a7ef6d33b141ce8fb1cec517faffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:31.033872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a6d23a71-ed96-4499-ba72-5e0eedade437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:31.185032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:31.226239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 829/1667] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:31.232413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:31.232619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:31.232996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:31.260954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:31.261540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:31.262201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:31.326420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] MFA Rules not processed for user `34a7ef6d33b141ce8fb1cec517faffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:31.334397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:31.363833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:31.370211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:31.370965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 839/1668] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:31.392721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:31.402928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:31.673508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff009-3e06-4d46-987e-e2457d4eb15d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:31.711452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:31.712423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:31.712869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:31.713375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:31.729381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 830/1669] 10.222.0.47 () {58 vars in 1015 bytes} [Sat Jun 4 09:26:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:31.749284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:31.759059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:31.774806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:31.776360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:31.798010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e342965-6c44-450b-bd82-0c8bd571fa28 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:31.800732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 840/1670] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:31.813435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b1ca1-0c6e-40f6-bdac-91cd3c8c4070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:31.813998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1ca1-0c6e-40f6-bdac-91cd3c8c4070 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:31.814336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1ca1-0c6e-40f6-bdac-91cd3c8c4070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:31.814613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1ca1-0c6e-40f6-bdac-91cd3c8c4070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:31.832272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b1ca1-0c6e-40f6-bdac-91cd3c8c4070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:31.845469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b1ca1-0c6e-40f6-bdac-91cd3c8c4070 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:31.866687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 831/1671] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:32.083992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:32.087064 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:32.313686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2b59-9d26-4d07-8a53-747123dcd7a4 service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:32.334743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:32.339524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:32.345826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:32.345826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] PATH_INFO: `/v3/projects/64e9b1acaed94e648a8c2debe67bee7c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:32.345826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e9b1acaed94e648a8c2debe67bee7c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:32.357182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:32.677678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e9b1acaed94e648a8c2debe67bee7c', 'request_id': 'req-74003eee-0a21-4184-8cf5-551a5a1b95a8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:32.702640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e9b1acaed94e648a8c2debe67bee7c', 'request_id': 'req-74003eee-0a21-4184-8cf5-551a5a1b95a8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:32.883283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e9b1acaed94e648a8c2debe67bee7c', 'request_id': 'req-74003eee-0a21-4184-8cf5-551a5a1b95a8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:32.914971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] The token cache is being invalidate because project 64e9b1acaed94e648a8c2debe67bee7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:32.917580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:32.922765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-74003eee-0a21-4184-8cf5-551a5a1b95a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:32.927867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 841/1672] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:32 2022] DELETE /identity/v3/projects/64e9b1acaed94e648a8c2debe67bee7c => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:32.933717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:32.935353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:32.936863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:32.987919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:32.989449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:33.181882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 832/1673] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:33.192988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:33.193785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:33.195284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:33.256114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:33.263893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:33.325904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 842/1674] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:33.332691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:33.365046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6042b62-4b5b-4d6c-9fbe-5aa13312fd52 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:33.385406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:33.386916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:33.389191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:33.389665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:33.400106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:33.404488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:33.406523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6acec1-d4dd-4957-8b87-a0202a579e99 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:33.461881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 833/1675] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:33.468175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:33.489115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269eebf-7e29-4ffa-8813-48497f28564d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:33.520471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:33.521229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:33.521516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:33.521796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:33.522861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:33.525437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:33.575143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:33.575813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:33.576831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:33.687972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] MFA Rules not processed for user `7b6676edfa984008b846589b2df9ef92`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:33.691248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:33.807386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 843/1676] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:33 2022] POST /identity/v3/projects => generated 373 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:26:33.811648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:33.816934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:33.950081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 834/1677] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:33.951835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:33.990918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e9212-9852-4193-8ace-c397e272c8aa None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:34.064247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab719beb-0210-46fe-8426-0085df000875 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:34.091146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:34.092917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:34.093196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:34.093474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] PATH_INFO: `/v3/projects/da46d50caf4d4767a4fe522d85bbe513` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:34.098618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da46d50caf4d4767a4fe522d85bbe513)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:34.102257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:34.195555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:34.197071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:34.197590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:34.198123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:34.199064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:34.201875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:34.373857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 835/1678] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:33 2022] POST /identity/v3/users => generated 287 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:34.375135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:34.376328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da46d50caf4d4767a4fe522d85bbe513', 'request_id': 'req-2335b59e-eb96-4585-bc8d-874d95d973d3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:34.389184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:34.404905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da46d50caf4d4767a4fe522d85bbe513', 'request_id': 'req-2335b59e-eb96-4585-bc8d-874d95d973d3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:34.564154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da46d50caf4d4767a4fe522d85bbe513', 'request_id': 'req-2335b59e-eb96-4585-bc8d-874d95d973d3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:34.598866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] The token cache is being invalidate because project da46d50caf4d4767a4fe522d85bbe513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:34.599171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:34.603828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:34.611101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 844/1679] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:33 2022] DELETE /identity/v3/projects/da46d50caf4d4767a4fe522d85bbe513 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:34.613269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:34.618949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:34.689351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddf18-73dd-4607-9167-07d1a8b4b12a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:34.856631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:34.925160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335b59e-eb96-4585-bc8d-874d95d973d3 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:34.942960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8ac20-829a-4e97-bad1-f92afc0bf572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:34.943798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8ac20-829a-4e97-bad1-f92afc0bf572 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:34.944088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8ac20-829a-4e97-bad1-f92afc0bf572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:34.944406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8ac20-829a-4e97-bad1-f92afc0bf572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:34.949398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8ac20-829a-4e97-bad1-f92afc0bf572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:34.951283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8ac20-829a-4e97-bad1-f92afc0bf572 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:34.972889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:34.974684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:34.976089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:34.978139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:34.980973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 845/1680] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:34.982672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:34.982964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:34.983252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:34.989370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:35.037724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:35.038869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:35.068956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 846/1681] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:35.069728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:35.092972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5a375-8162-4e31-a563-3053d845f995 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:35.116191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18d0ef-60e1-40c5-8ba9-515b918f902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:35.116947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d0ef-60e1-40c5-8ba9-515b918f902b admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:35.117972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d0ef-60e1-40c5-8ba9-515b918f902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:35.118243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d0ef-60e1-40c5-8ba9-515b918f902b admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:35.119024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18d0ef-60e1-40c5-8ba9-515b918f902b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:35.120833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18d0ef-60e1-40c5-8ba9-515b918f902b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:35.151042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 847/1682] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:35.152569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:35.152909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:35.153312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:35.222396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:35.229833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:35.270158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 848/1683] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:35.270158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:35.332231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:35.342811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:35.344518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902e853-842c-4454-a476-f7b9beb27b21 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:35.383623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359433c-6e90-4e01-8ffc-8ccd0cda4301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:35.385176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359433c-6e90-4e01-8ffc-8ccd0cda4301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:35.388709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 836/1684] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 1014 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:35.391703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:35.397807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359433c-6e90-4e01-8ffc-8ccd0cda4301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:35.400118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359433c-6e90-4e01-8ffc-8ccd0cda4301 admin admin] PATH_INFO: `/v3/projects/d9f10bade7f942dfb6deaec35c6fd9c0/users/d76295c1980544a4b41c20c6ef7d8157/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:35.463024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8784-cdd8-45bc-88f9-d8eb0b90cd36 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:35.467491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359433c-6e90-4e01-8ffc-8ccd0cda4301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f10bade7f942dfb6deaec35c6fd9c0, user_id=d76295c1980544a4b41c20c6ef7d8157, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:35.482863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:35.484007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359433c-6e90-4e01-8ffc-8ccd0cda4301 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:35.484165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:35.490792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:35.491386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:35.493024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:35.498382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:35.499760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:35.563097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 837/1685] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:35.568525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:35.602337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f7d29-1092-4950-b5b3-3ea3ac849237 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:35.629422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:35.630576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:35.631076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:35.631585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:35.634777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:35.638705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:35.689342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 849/1686] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:35 2022] PUT /identity/v3/projects/d9f10bade7f942dfb6deaec35c6fd9c0/users/d76295c1980544a4b41c20c6ef7d8157/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:35.692588 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:35.693198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:35.693698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:35.772582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] MFA Rules not processed for user `d76295c1980544a4b41c20c6ef7d8157`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:35.774037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:35.894735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:35.932860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 838/1687] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:26:35.934759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:36.035776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 850/1688] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:36.076952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9191d-f44e-4f1d-afaa-ac99623c8abc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:36.097858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:36.105755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:36.105755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:36.105755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:36.105755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:36.108683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:36.349246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:36.349470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 839/1689] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:35 2022] POST /identity/v3/users => generated 292 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:36.539820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cfde4-cc1d-46c1-a9f0-44fe2b7e191a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:36.545308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:36.546459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:36.546459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:36.546840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:36.547772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:36.549936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:36.565287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 851/1690] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:36.571399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:36.593255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c142bf6-6a6c-4a20-ae65-2b4d7b1521f1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:36.599865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:36.601022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:36.601412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:36.601922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:36.602931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:36.605711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:36.649573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 840/1691] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:36.656571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:36.703702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaecab0-204d-4b44-9029-851d365c4b54 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:36.739025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57be03-4286-48a4-a537-1abeae04a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:36.747922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57be03-4286-48a4-a537-1abeae04a02e admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:36.748941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57be03-4286-48a4-a537-1abeae04a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:36.749808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57be03-4286-48a4-a537-1abeae04a02e admin admin] PATH_INFO: `/v3/projects/8bbec7a53e724cf0990f924b81046394/users/c6a105dcd4e04bde888e62307a38586a/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:36.811597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57be03-4286-48a4-a537-1abeae04a02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bbec7a53e724cf0990f924b81046394, user_id=c6a105dcd4e04bde888e62307a38586a, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:36.817889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57be03-4286-48a4-a537-1abeae04a02e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:36.899591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:36.913460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:37.072061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 852/1692] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:36 2022] PUT /identity/v3/projects/8bbec7a53e724cf0990f924b81046394/users/c6a105dcd4e04bde888e62307a38586a/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:37.085885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:37.086238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:37.086528 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:37.212629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] MFA Rules not processed for user `c6a105dcd4e04bde888e62307a38586a`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:37.214228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:37.395269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b408f8-37d1-494c-9340-9fd2b83e8509 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:37.412360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 853/1693] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:37.422786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:37.425500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:37.449722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:37.549475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ee1ea-ad1b-4e47-b11b-52fcd1125122 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:37.556436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93edcfb-785c-4151-b596-6f704d951149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:37.556983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93edcfb-785c-4151-b596-6f704d951149 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:37.557368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93edcfb-785c-4151-b596-6f704d951149 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:37.557696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93edcfb-785c-4151-b596-6f704d951149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:37.570577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93edcfb-785c-4151-b596-6f704d951149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:37.573139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93edcfb-785c-4151-b596-6f704d951149 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:37.607171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 854/1694] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:37.745558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84e370-28dd-4282-a433-2fc31db81003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:37.759917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e370-28dd-4282-a433-2fc31db81003 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:37.766669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e370-28dd-4282-a433-2fc31db81003 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:37.767025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e370-28dd-4282-a433-2fc31db81003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:38.210821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84e370-28dd-4282-a433-2fc31db81003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:38.214477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84e370-28dd-4282-a433-2fc31db81003 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:38.268311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 841/1695] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 1377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:38.783795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:38.817474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:38.818061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:38.818061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:38.845722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] MFA Rules not processed for user `34a7ef6d33b141ce8fb1cec517faffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:38.847570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:39.148248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 855/1696] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:39.262206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8beb44-03ca-4366-8556-1cbf19e3f019 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:39.262518 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8beb44-03ca-4366-8556-1cbf19e3f019 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:39.262772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8beb44-03ca-4366-8556-1cbf19e3f019 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:39.265136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 842/1697] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:26:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:39.353735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:39.404226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fc851-b87a-45b4-8b14-86f70b1e415a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:39.404565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fc851-b87a-45b4-8b14-86f70b1e415a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:39.405135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fc851-b87a-45b4-8b14-86f70b1e415a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:39.530184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8c0fc851-b87a-45b4-8b14-86f70b1e415a None None] MFA Rules not processed for user `aeffa44fe38a4204ae21b52fe827cedc`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:39.531663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8c0fc851-b87a-45b4-8b14-86f70b1e415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:39.649594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42eb702-9353-49cd-b328-718df52ff117 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:39.671388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:39.677150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:39.680535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:39.682098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] PATH_INFO: `/v3/users/d76295c1980544a4b41c20c6ef7d8157` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:39.689966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76295c1980544a4b41c20c6ef7d8157)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:39.695644 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:39.836381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 843/1698] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:39.953028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936030da-f36b-4e75-bc14-8b9167488306 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:39.953401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936030da-f36b-4e75-bc14-8b9167488306 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:39.953690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936030da-f36b-4e75-bc14-8b9167488306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:40.031062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-936030da-f36b-4e75-bc14-8b9167488306 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:40.038844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-936030da-f36b-4e75-bc14-8b9167488306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:40.065502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 844/1699] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:26:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:40.217966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76295c1980544a4b41c20c6ef7d8157'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:40.241446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76295c1980544a4b41c20c6ef7d8157'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:40.243579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76295c1980544a4b41c20c6ef7d8157'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:40.285505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76295c1980544a4b41c20c6ef7d8157'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:40.287630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76295c1980544a4b41c20c6ef7d8157'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:40.441295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:40.501416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 856/1700] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:39 2022] DELETE /identity/v3/users/d76295c1980544a4b41c20c6ef7d8157 => generated 0 bytes in 1148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:40.507417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:40.508102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:40.508645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:40.567219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:40.570475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:40.836335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 845/1701] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:40.904807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:40.913989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:41.045165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:41.047577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:41.291687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ae996-aef6-4dd7-a029-1231f5bf549c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:41.325297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:41.326181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:41.330166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:41.334972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:41.346180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:41.379863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ae191-4692-4a49-ab30-996e4e19b91e admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:41.410118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:41.410693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:41.411167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:41.411600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:41.458977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:41.463678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:41.512713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 857/1702] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:41.537027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:41.560958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21c7f8-c38f-4a6b-9c88-d521a9afe7f1 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:41.635576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 846/1703] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:42.080476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:42.110121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69635b17-4287-4730-839e-5cc901bca58f service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:42.130846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba11669-f418-43fd-bcdb-b604ab8326c6 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:42.131457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba11669-f418-43fd-bcdb-b604ab8326c6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:42.132417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba11669-f418-43fd-bcdb-b604ab8326c6 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:42.134618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:42.135597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:42.136306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 847/1704] 10.222.0.46 () {54 vars in 912 bytes} [Sat Jun 4 09:26:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:42.137977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:42.139132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:42.144811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:42.147953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de4c52-b187-4408-b5b4-8dfe4159236b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:42.148597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de4c52-b187-4408-b5b4-8dfe4159236b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:42.149074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de4c52-b187-4408-b5b4-8dfe4159236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:42.213947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:42.215649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:42.253841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-96de4c52-b187-4408-b5b4-8dfe4159236b None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:42.258135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-96de4c52-b187-4408-b5b4-8dfe4159236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:42.275431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 858/1705] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:42.308409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 848/1706] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:26:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:42.443989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:42.500512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff0b65-5130-4593-904d-65eb463d7ee3 service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:42.522516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfd916-1866-4733-9a0e-c2f3c63ed924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:42.526987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfd916-1866-4733-9a0e-c2f3c63ed924 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:42.527392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfd916-1866-4733-9a0e-c2f3c63ed924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:42.531341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfd916-1866-4733-9a0e-c2f3c63ed924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:42.580936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfd916-1866-4733-9a0e-c2f3c63ed924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:42.598714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfd916-1866-4733-9a0e-c2f3c63ed924 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:42.690409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 859/1707] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:43.326971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:44.902573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:44.903844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:44.904372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:44.953427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] MFA Rules not processed for user `7b6676edfa984008b846589b2df9ef92`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:44.954910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:45.102576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 849/1708] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:46.635157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b6af-4c81-4b72-bc2c-e900cf7f8912 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:46.640506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b6af-4c81-4b72-bc2c-e900cf7f8912 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:46.640506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b6af-4c81-4b72-bc2c-e900cf7f8912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:46.668895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:46.671945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:46.733264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb2260-272b-43e5-a5a2-5cd55fdcbf21 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:46.758176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:46.763677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:46.764422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:46.764962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] PATH_INFO: `/v3/projects/d9f10bade7f942dfb6deaec35c6fd9c0` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:46.766100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b445b6af-4c81-4b72-bc2c-e900cf7f8912 None None] MFA Rules not processed for user `ac869ff528aa49989c88b3a545ba9cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:46.768233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b445b6af-4c81-4b72-bc2c-e900cf7f8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:46.769855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f10bade7f942dfb6deaec35c6fd9c0)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:46.778260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:46.981288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f10bade7f942dfb6deaec35c6fd9c0', 'request_id': 'req-ee090159-212b-4595-aed3-fd3f5a360250'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:46.997457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f10bade7f942dfb6deaec35c6fd9c0', 'request_id': 'req-ee090159-212b-4595-aed3-fd3f5a360250'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:47.156261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 860/1709] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:47.157691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f10bade7f942dfb6deaec35c6fd9c0', 'request_id': 'req-ee090159-212b-4595-aed3-fd3f5a360250'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:47.184795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] The token cache is being invalidate because project d9f10bade7f942dfb6deaec35c6fd9c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:47.186131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:47.188641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ee090159-212b-4595-aed3-fd3f5a360250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:47.194193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 850/1710] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:46 2022] DELETE /identity/v3/projects/d9f10bade7f942dfb6deaec35c6fd9c0 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:47.238885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:47.240050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:47.240909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:47.304352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:47.307450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:47.510523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 861/1711] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:47.525847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:47.526251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:47.526699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:47.561830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:47.618006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:47.621033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:47.668581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 851/1712] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:47.669962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:47.705840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:47.709747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:47.723446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe4df1-90a5-4d1e-baa0-dea9dbd78a26 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:47.738532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:47.740143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:47.740947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:47.741644 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:47.754227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:47.764489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:47.782372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:47.788415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17705cc-a7b1-4448-9b1d-a640496c56dd None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:47.816536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 862/1713] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:47.822673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:47.828873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:47.830825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:47.836800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:47.838904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] PATH_INFO: `/v3/users/afc2a5334f7046769f70b9598f589407` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:47.851475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc2a5334f7046769f70b9598f589407)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:47.853810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba095-8d44-45a3-a321-a69cc2f60831 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:47.866680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:47.881527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:47.887310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:47.887310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:47.887310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:47.887310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:47.887939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:48.091543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 863/1714] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:47 2022] POST /identity/v3/projects => generated 403 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:48.092995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:48.212572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:48.305639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc2a5334f7046769f70b9598f589407'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:48.317217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc2a5334f7046769f70b9598f589407'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:48.388726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc2a5334f7046769f70b9598f589407'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:48.464619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc2a5334f7046769f70b9598f589407'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:48.469148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-40b29520-276d-415e-b89f-a29101b50ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc2a5334f7046769f70b9598f589407'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:48.522517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86694c6-b1ee-4620-881c-1ed39d896f58 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:48.545939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:48.548533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:48.551002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:48.551623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:48.553736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:48.559411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:48.883465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 852/1715] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:47 2022] DELETE /identity/v3/users/afc2a5334f7046769f70b9598f589407 => generated 0 bytes in 1180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:48.890921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:48.896782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:48.898405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:48.946901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:48.946901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:48.992256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 864/1716] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:48 2022] POST /identity/v3/users => generated 302 bytes in 900 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:48.994137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:49.004997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:49.049083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d8d7-3550-4eea-98ff-2301b5e8d70a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:49.178799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 853/1717] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:49.233579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:49.236811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:49.335284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:49.339622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:49.342049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:49.349051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:49.350495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:49.358435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:49.404623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 865/1718] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:49.407090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:49.430091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7447363-2026-4cc5-ba7b-18e2f923dfd0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:49.444995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:49.444995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:49.444995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:49.445589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:49.447463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:49.451225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:49.484901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55531f12-a515-42cb-84ad-7a7f49789e47 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:49.489246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 866/1719] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:49.494510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:49.500074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bb7c0-7a32-478e-aefc-260f24aa7cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:49.500074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bb7c0-7a32-478e-aefc-260f24aa7cbc service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:49.500074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bb7c0-7a32-478e-aefc-260f24aa7cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:49.500074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bb7c0-7a32-478e-aefc-260f24aa7cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:49.523356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe660bf9-6a1a-4784-8420-39237ded3124 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:49.527127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bb7c0-7a32-478e-aefc-260f24aa7cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:49.528794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bb7c0-7a32-478e-aefc-260f24aa7cbc service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:49.541231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:49.541586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 854/1720] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:49.544820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:49.545107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:49.545406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] PATH_INFO: `/v3/projects/4cd397ed9d7b4db9b1a818842803d8df/users/00f0e14454144adbb37c4799f35c6415/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:49.574590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd397ed9d7b4db9b1a818842803d8df, user_id=00f0e14454144adbb37c4799f35c6415, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:49.577657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:49.764840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 867/1721] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:49 2022] PUT /identity/v3/projects/4cd397ed9d7b4db9b1a818842803d8df/users/00f0e14454144adbb37c4799f35c6415/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:49.770926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:49.771414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:49.771805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:49.806383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:49.808401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:49.942771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 855/1722] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:50.025948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:50.029688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:50.248251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c56a08-9c59-49fd-8230-a6cba45a0a7c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:50.281807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc53fe-4727-4b2c-a9cd-78d2b28cb9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:50.289094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc53fe-4727-4b2c-a9cd-78d2b28cb9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:50.292877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc53fe-4727-4b2c-a9cd-78d2b28cb9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:50.298741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc53fe-4727-4b2c-a9cd-78d2b28cb9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:50.319560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:50.360935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc53fe-4727-4b2c-a9cd-78d2b28cb9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:50.366408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc53fe-4727-4b2c-a9cd-78d2b28cb9a8 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:50.440769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 868/1723] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:51.194111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:51.670849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:51.675137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:51.698516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403816b-18c2-44e1-a8d4-8314e2fcb065 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:51.735027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:51.744563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:51.744563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:51.744563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] PATH_INFO: `/v3/projects/e92b4e1df142437f9a01f58f7e638aa0` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:51.747601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e92b4e1df142437f9a01f58f7e638aa0)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:51.756316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:51.817644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:52.050434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e92b4e1df142437f9a01f58f7e638aa0', 'request_id': 'req-26bbbb83-b2ad-4331-b338-5748c0cd2f47'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:52.076264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e92b4e1df142437f9a01f58f7e638aa0', 'request_id': 'req-26bbbb83-b2ad-4331-b338-5748c0cd2f47'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:52.185897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0717b3-1d9a-4dee-a777-40c6d22f02b5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:52.186583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0717b3-1d9a-4dee-a777-40c6d22f02b5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:52.187862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0717b3-1d9a-4dee-a777-40c6d22f02b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:52.221419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7f0717b3-1d9a-4dee-a777-40c6d22f02b5 None None] MFA Rules not processed for user `aeffa44fe38a4204ae21b52fe827cedc`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:52.225063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7f0717b3-1d9a-4dee-a777-40c6d22f02b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:52.238275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e92b4e1df142437f9a01f58f7e638aa0', 'request_id': 'req-26bbbb83-b2ad-4331-b338-5748c0cd2f47'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:52.320024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] The token cache is being invalidate because project e92b4e1df142437f9a01f58f7e638aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:52.325140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:52.326839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:52.340923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 856/1724] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:51 2022] DELETE /identity/v3/projects/e92b4e1df142437f9a01f58f7e638aa0 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:52.343647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:52.347650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:52.439438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 869/1725] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:52.694001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbb83-b2ad-4331-b338-5748c0cd2f47 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:52.718675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:52.720312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:52.722031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:52.722432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] PATH_INFO: `/v3/users/a51b41b3119248f39ff2c32d8e532b4e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:52.724054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51b41b3119248f39ff2c32d8e532b4e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:52.726279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:53.105529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51b41b3119248f39ff2c32d8e532b4e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:53.106271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51b41b3119248f39ff2c32d8e532b4e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:53.138916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51b41b3119248f39ff2c32d8e532b4e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:53.170054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51b41b3119248f39ff2c32d8e532b4e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:53.174791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-20247250-a1e8-4713-9daa-9b0b44f2f26b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51b41b3119248f39ff2c32d8e532b4e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:53.325337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 857/1726] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:52 2022] DELETE /identity/v3/users/a51b41b3119248f39ff2c32d8e532b4e => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:53.831122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215860b-b8ff-462f-b89f-1a56aef000d2 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:53.834315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215860b-b8ff-462f-b89f-1a56aef000d2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:53.837187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215860b-b8ff-462f-b89f-1a56aef000d2 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:53.843426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 870/1727] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:26:53 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:53.864481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:53.865899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:53.869698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:53.902639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:53.905928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:54.028623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:54.051120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 858/1728] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:26:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:54.134760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215860b-b8ff-462f-b89f-1a56aef000d2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:54.137040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2215860b-b8ff-462f-b89f-1a56aef000d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:54.279712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215860b-b8ff-462f-b89f-1a56aef000d2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:54.290336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:54.291294 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:54.291960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:54.292518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] PATH_INFO: `/v3/users/c6a105dcd4e04bde888e62307a38586a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:54.296762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a105dcd4e04bde888e62307a38586a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:54.299249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:54.504336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:54.507934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:54.534981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af510657-7abe-4e1f-8033-cfb5e1ec3f60 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:54.542140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:54.543170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:54.543910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:54.544201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:54.558030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:54.559381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:54.574855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 859/1729] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:54.779024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:54.781704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:54.825618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a105dcd4e04bde888e62307a38586a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:54.851288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a105dcd4e04bde888e62307a38586a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:54.852772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e34f-4971-4a8b-8ff7-cc41bd1f6651 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:54.853274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a105dcd4e04bde888e62307a38586a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:54.878395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:54.883970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a105dcd4e04bde888e62307a38586a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:54.885361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:54.886073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:54.886900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] PATH_INFO: `/v3/projects/23f464f9a78e4698a2002c96c48867cb` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:54.894030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-180bc396-76ba-4533-8c5e-6198aeb4da9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a105dcd4e04bde888e62307a38586a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:54.901882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f464f9a78e4698a2002c96c48867cb)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:54.910022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:55.058932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 871/1730] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:26:54 2022] DELETE /identity/v3/users/c6a105dcd4e04bde888e62307a38586a => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:55.061158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057c966-0526-4684-9aad-4f9fdb10a912 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.061988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057c966-0526-4684-9aad-4f9fdb10a912 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.062654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057c966-0526-4684-9aad-4f9fdb10a912 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.067370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 872/1731] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:26:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:26:55.070921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34e89-2afb-4083-91d9-bfdc345ef2ca None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.077039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34e89-2afb-4083-91d9-bfdc345ef2ca None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.081285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34e89-2afb-4083-91d9-bfdc345ef2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.135209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-62d34e89-2afb-4083-91d9-bfdc345ef2ca None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:55.137896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-62d34e89-2afb-4083-91d9-bfdc345ef2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.250560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f464f9a78e4698a2002c96c48867cb', 'request_id': 'req-d2717493-c728-494c-92ab-496f19783a72'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:55.272589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f464f9a78e4698a2002c96c48867cb', 'request_id': 'req-d2717493-c728-494c-92ab-496f19783a72'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:55.288992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 873/1732] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:55.294397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b06a-18ea-4f75-a19c-5912c41c19b8 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.294966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b06a-18ea-4f75-a19c-5912c41c19b8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.295564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b06a-18ea-4f75-a19c-5912c41c19b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.340199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d700b06a-18ea-4f75-a19c-5912c41c19b8 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:55.343129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d700b06a-18ea-4f75-a19c-5912c41c19b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.427308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f464f9a78e4698a2002c96c48867cb', 'request_id': 'req-d2717493-c728-494c-92ab-496f19783a72'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:55.482054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] The token cache is being invalidate because project 23f464f9a78e4698a2002c96c48867cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:55.482731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:55.498511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:55.498511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 860/1733] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:54 2022] DELETE /identity/v3/projects/23f464f9a78e4698a2002c96c48867cb => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:55.498511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:55.503155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.561865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 874/1734] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:55.564820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.566070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.566913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.621105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:55.623975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.690689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2717493-c728-494c-92ab-496f19783a72 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:55.716241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:55.717243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.718129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.718416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.725507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.856587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 875/1735] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:55.859169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:55.867067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.903563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80464f9c-52fb-468f-b994-2e95ed849bd3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:55.917950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:55.920105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.921144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.921593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.927766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:55.929287 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b315408-72e0-4724-83c5-1c9f2cc649c0 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:55.941036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:55.941036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:55.941749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 861/1736] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:55.943727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:55.944189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:55.944651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:55.966243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:55.966873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 876/1737] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:26:55.967970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:55.977719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 862/1738] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:55.983111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:55.995020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:55.999701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb023-2839-4f33-af08-359c02925436 service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:56.006598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:56.008932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.009312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.009976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.011603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:26:56.012186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:56.014318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:56.041737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:56.042248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 877/1739] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:26:56.053529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db8f83-fd40-4e3f-8293-fc4f61b3d1d1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:56.059221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:56.060198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.060686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.061726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.063234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:56.065482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:56.244637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 863/1740] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:56 2022] POST /identity/v3/projects => generated 363 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:56.250331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:56.268550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4796b05-18e0-43ed-a620-f7535ef13df8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:56.419527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:56.423953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.424623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.425180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.427500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:56.431725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:56.686937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 878/1741] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:56 2022] POST /identity/v3/users => generated 282 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:56.687250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:56.715248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dd8e0-d369-4377-b085-a962f4e4f37c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:56.721380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:56.722336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.722732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.723147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.724021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:56.733711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:56.740653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 864/1742] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:56.747329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:56.762789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac95d8-6fc0-4a91-9255-8c3cbb972a8a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:56.776632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a67c35-31e9-462b-a10d-334041499ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:56.777530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a67c35-31e9-462b-a10d-334041499ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.777882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a67c35-31e9-462b-a10d-334041499ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.778232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a67c35-31e9-462b-a10d-334041499ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.779103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a67c35-31e9-462b-a10d-334041499ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:56.784876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a67c35-31e9-462b-a10d-334041499ae1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:56.808652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 879/1743] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:26:56.814383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:56.828949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0f5eb-dd64-4934-822b-2923ca5506e6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:56.842851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c555201-82d8-4bf1-b49c-06747bf48b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:56.845574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c555201-82d8-4bf1-b49c-06747bf48b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.845915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c555201-82d8-4bf1-b49c-06747bf48b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.846164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c555201-82d8-4bf1-b49c-06747bf48b74 admin admin] PATH_INFO: `/v3/projects/edacca4533664b77bc9a033e11961002/users/1c21ce575ee240a6b8dba5dfa36ab797/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.892977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c555201-82d8-4bf1-b49c-06747bf48b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edacca4533664b77bc9a033e11961002, user_id=1c21ce575ee240a6b8dba5dfa36ab797, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:56.895870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c555201-82d8-4bf1-b49c-06747bf48b74 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:56.907224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:56.907787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:56.908221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:56.952619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] MFA Rules not processed for user `00f0e14454144adbb37c4799f35c6415`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:56.954443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:57.070439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 880/1744] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:26:57.071796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:57.098391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 865/1745] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:56 2022] PUT /identity/v3/projects/edacca4533664b77bc9a033e11961002/users/1c21ce575ee240a6b8dba5dfa36ab797/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:57.105922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:57.124814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:57.132733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:57.133858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:57.157758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2530197-7ab8-44ac-a8e7-55568e0249f2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:57.284466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] MFA Rules not processed for user `1c21ce575ee240a6b8dba5dfa36ab797`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:57.286566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:57.300846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:57.302570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:57.303389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:57.305143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:57.307250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:57.316569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:57.535201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 866/1746] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:57.543557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:57.610126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ecc5f-6c6f-4baf-af56-cd6ad3b4d3e5 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:57.610126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:57.610126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:57.610126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:57.610821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:57.610821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:57.610821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:57.724053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 881/1747] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:57 2022] POST /identity/v3/projects => generated 405 bytes in 639 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:26:57.729173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:57.903570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89c8c-45b7-4853-8840-3e5cc6c1613c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:57.916546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:57.918236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:57.923310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:57.923310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:57.923310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:57.926575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:57.981500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 867/1748] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:26:57 2022] POST /identity/v3/projects => generated 367 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:57.987629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.147513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d03ef-a609-4a9f-b6ea-709ffdfd0eda admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.162376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 882/1749] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:57 2022] POST /identity/v3/users => generated 309 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:26:58.167105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.170963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:58.172661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:58.173471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:58.174239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:58.175855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:58.180086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:58.185004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c446b5-2777-4037-8495-e7bec48b9099 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.197219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:58.197972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:58.198259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:58.198539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:58.206404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:58.206404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:58.216151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 883/1750] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:26:58.224052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.262426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9ead8-b1d5-41e6-bc81-bd80120cfb66 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.272759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:58.274528 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:58.275664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:58.276366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] PATH_INFO: `/v3/projects/a3cf9f1769d34933b2ad6929059c7478/users/679aa395b2a1436a91d98291916ee722/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:58.306927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cf9f1769d34933b2ad6929059c7478, user_id=679aa395b2a1436a91d98291916ee722, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:58.311789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:58.326896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 868/1751] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:26:57 2022] POST /identity/v3/users => generated 290 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:26:58.330440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.341722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c527c-b8dd-44df-b027-3ef534cd4f2b admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.348929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:58.349282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:58.349433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:58.349698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:58.350553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:58.352399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:58.365572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 869/1752] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:58.370176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.382635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f62e2-d3d0-490d-953a-f6d0e1415133 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.391864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:58.393101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:58.393540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:58.393987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] PATH_INFO: `/v3/projects/ac7132cb59ee46f481f7fae5063518bd/users/a8bc95157be2461b82fb23492fd5816e/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:58.425038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7132cb59ee46f481f7fae5063518bd, user_id=a8bc95157be2461b82fb23492fd5816e, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:58.425038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:58.592946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 884/1753] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:58 2022] PUT /identity/v3/projects/a3cf9f1769d34933b2ad6929059c7478/users/679aa395b2a1436a91d98291916ee722/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:58.599879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.654254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 870/1754] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:58 2022] PUT /identity/v3/projects/ac7132cb59ee46f481f7fae5063518bd/users/a8bc95157be2461b82fb23492fd5816e/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:26:58.663029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:58.720444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:58.790309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffc07a-6940-4ab2-ad61-f4496595e2c0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.985360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83445c61-1fab-47f8-8f4f-0cb40b88a813 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:58.993331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:26:59.015136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:59.019628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.020911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.022117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.023517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:59.026782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:59.032779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:59.033932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.034420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.034914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.036384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:59.039634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:59.079117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 871/1755] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:59.087071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:59.094160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 885/1756] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:26:59.098472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:59.119622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728d95d-da30-477d-83c6-5f8505ce3654 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:59.128118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558a6b4-d320-496e-804f-1ace574b407d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:59.131411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bdcd-5b8e-41cd-b0a9-d22628ed98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:59.133746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bdcd-5b8e-41cd-b0a9-d22628ed98c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.134873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bdcd-5b8e-41cd-b0a9-d22628ed98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.135818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bdcd-5b8e-41cd-b0a9-d22628ed98c1 admin admin] PATH_INFO: `/v3/projects/ac7132cb59ee46f481f7fae5063518bd/users/a8bc95157be2461b82fb23492fd5816e/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.140923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bdcd-5b8e-41cd-b0a9-d22628ed98c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7132cb59ee46f481f7fae5063518bd, user_id=a8bc95157be2461b82fb23492fd5816e, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:59.149739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bdcd-5b8e-41cd-b0a9-d22628ed98c1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:59.151437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:59.152479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.152767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.153049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] PATH_INFO: `/v3/projects/a3cf9f1769d34933b2ad6929059c7478/users/679aa395b2a1436a91d98291916ee722/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.156665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cf9f1769d34933b2ad6929059c7478, user_id=679aa395b2a1436a91d98291916ee722, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:59.162208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:59.186346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 886/1757] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:59 2022] PUT /identity/v3/projects/a3cf9f1769d34933b2ad6929059c7478/users/679aa395b2a1436a91d98291916ee722/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:59.187499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:59.193519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:59.206974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 872/1758] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:26:59 2022] PUT /identity/v3/projects/ac7132cb59ee46f481f7fae5063518bd/users/a8bc95157be2461b82fb23492fd5816e/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:59.219714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.220627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.221235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.362528 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9d7fa-d218-425e-9d02-c788145597b7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:59.374677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:59.374677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.374677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.374677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] PATH_INFO: `/v3/projects/8bbec7a53e724cf0990f924b81046394` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.376700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bbec7a53e724cf0990f924b81046394)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:59.379587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:59.379940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] MFA Rules not processed for user `a8bc95157be2461b82fb23492fd5816e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:59.381513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:59.533977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 873/1759] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:26:59.545646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:26:59.547531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:59.592028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bbec7a53e724cf0990f924b81046394', 'request_id': 'req-119b355a-11c9-4152-a889-a0b60192c1f0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:59.604816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bbec7a53e724cf0990f924b81046394', 'request_id': 'req-119b355a-11c9-4152-a889-a0b60192c1f0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:59.641060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd311fd-2637-4572-8f40-a519ef40eca0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:26:59.708137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268361f-a6fe-4921-883e-fbb06369b7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:26:59.709010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268361f-a6fe-4921-883e-fbb06369b7ab service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.709305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268361f-a6fe-4921-883e-fbb06369b7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.709590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268361f-a6fe-4921-883e-fbb06369b7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.803544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bbec7a53e724cf0990f924b81046394', 'request_id': 'req-119b355a-11c9-4152-a889-a0b60192c1f0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:59.828364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] The token cache is being invalidate because project 8bbec7a53e724cf0990f924b81046394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:26:59.828700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:59.830830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-119b355a-11c9-4152-a889-a0b60192c1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:26:59.834643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 887/1760] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:26:59 2022] DELETE /identity/v3/projects/8bbec7a53e724cf0990f924b81046394 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:26:59.840101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.841044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.841342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.855500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268361f-a6fe-4921-883e-fbb06369b7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:26:59.859431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268361f-a6fe-4921-883e-fbb06369b7ab service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:26:59.884414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 874/1761] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:26:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:26:59.885656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:26:59.887063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:26:59.978757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:26:59.978973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:26:59.979355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:26:59.981306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 888/1762] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:00.005865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:00.008154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:00.019775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 875/1763] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:00.024452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:00.037797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a413d40-a257-4947-9d8e-c42030153363 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:00.044011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:00.045404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:00.045759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:00.046142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:00.047218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:00.047660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:00.050007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:00.073495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:00.074839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 889/1764] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:00.102550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f9ca1-eb05-460f-8773-157fa8992863 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:00.111027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:00.111720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:00.112006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:00.112306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:00.113090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:00.115499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:00.270122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:00.289870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 876/1765] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:00 2022] POST /identity/v3/projects => generated 361 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:00.356615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09935864-c273-4dee-a515-e6deebb9aa88 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:00.514904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:00.516018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:00.516565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:00.517043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:00.518340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:00.522712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:00.780962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 890/1766] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:00 2022] POST /identity/v3/users => generated 281 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:00.783032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:00.802088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd80cf-3c15-4860-8b8f-da3b419d1b00 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:00.812263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:00.813579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:00.814169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:00.814746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:00.816655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:00.820092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:00.847746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 877/1767] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:00.850619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:00.884264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eedae0-6b32-460c-8c5e-203994796da1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:00.908244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a34244-e782-4319-9398-6885d94a580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:00.909189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a34244-e782-4319-9398-6885d94a580c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:00.909666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a34244-e782-4319-9398-6885d94a580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:00.910119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a34244-e782-4319-9398-6885d94a580c admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:00.911414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a34244-e782-4319-9398-6885d94a580c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:00.924601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a34244-e782-4319-9398-6885d94a580c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:00.987065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 891/1768] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:00.993837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:01.049737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40414d6b-d791-4bec-9a25-d56a58f92ce6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:01.066422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:01.071264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:01.071681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:01.074952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] PATH_INFO: `/v3/projects/ef5f14d7e8084ef68b9bb6820272e787/users/be4941c6024b47ceae6fede5c13a7704/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:01.142671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5f14d7e8084ef68b9bb6820272e787, user_id=be4941c6024b47ceae6fede5c13a7704, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:01.154013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:01.318512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 878/1769] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:00 2022] PUT /identity/v3/projects/ef5f14d7e8084ef68b9bb6820272e787/users/be4941c6024b47ceae6fede5c13a7704/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:01.349789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:01.351794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:01.352680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:01.430128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:01.486112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] MFA Rules not processed for user `be4941c6024b47ceae6fede5c13a7704`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:01.506152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:01.709078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 892/1770] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:01.758457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:01.762517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:01.907697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b44853-31ea-44e8-894d-c7b3f8e70c9b admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:01.917703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca67936-3112-4c6b-b33b-497fe1fa2e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:01.919456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67936-3112-4c6b-b33b-497fe1fa2e5d service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:01.919908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67936-3112-4c6b-b33b-497fe1fa2e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:01.920204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67936-3112-4c6b-b33b-497fe1fa2e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:01.936905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca67936-3112-4c6b-b33b-497fe1fa2e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:01.939625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca67936-3112-4c6b-b33b-497fe1fa2e5d service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:01.958696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 879/1771] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:01.986969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:02.173739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17cd50-86db-4bc4-b1ae-cf52827ae01c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:02.179488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:02.180592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:02.180796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:02.181050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] PATH_INFO: `/v3/users/be4941c6024b47ceae6fede5c13a7704` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:02.182556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4941c6024b47ceae6fede5c13a7704)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:02.184648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:02.706978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4941c6024b47ceae6fede5c13a7704'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:02.723841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4941c6024b47ceae6fede5c13a7704'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:02.724906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4941c6024b47ceae6fede5c13a7704'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:02.744546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4941c6024b47ceae6fede5c13a7704'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:02.745531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-50b156ed-43c6-4338-83ac-8fd92ff3bd73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4941c6024b47ceae6fede5c13a7704'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:02.875210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 893/1772] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:01 2022] DELETE /identity/v3/users/be4941c6024b47ceae6fede5c13a7704 => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:02.882808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:02.883623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:02.883763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:02.938104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:02.939635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:03.076941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 880/1773] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:03.124463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56185d1-4780-45d9-bcfc-a75b39b79879 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:03.125694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56185d1-4780-45d9-bcfc-a75b39b79879 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:03.126002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56185d1-4780-45d9-bcfc-a75b39b79879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:03.179292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:03.182310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:03.201003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a56185d1-4780-45d9-bcfc-a75b39b79879 None None] MFA Rules not processed for user `679aa395b2a1436a91d98291916ee722`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:03.202299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a56185d1-4780-45d9-bcfc-a75b39b79879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:03.320855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205a67f-d561-4535-b845-790ae9e3b63f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:03.330165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:03.331191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:03.331860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:03.332393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:03.346914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:03.349105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:03.364551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 881/1774] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:03.369521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 894/1775] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:03.424544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:04.511836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:04.901485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:04.905666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:04.964996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8d90c-5f3d-4d61-8ebf-b1aebb84e8ab service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:04.971618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:04.972866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:04.973328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:04.973758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] PATH_INFO: `/v3/projects/ef5f14d7e8084ef68b9bb6820272e787` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:04.976014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef5f14d7e8084ef68b9bb6820272e787)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:04.978359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:05.116783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef5f14d7e8084ef68b9bb6820272e787', 'request_id': 'req-64b1d492-ccd5-4e1a-913c-dfb0cf608482'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:05.131673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef5f14d7e8084ef68b9bb6820272e787', 'request_id': 'req-64b1d492-ccd5-4e1a-913c-dfb0cf608482'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:05.256600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef5f14d7e8084ef68b9bb6820272e787', 'request_id': 'req-64b1d492-ccd5-4e1a-913c-dfb0cf608482'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:05.279729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] The token cache is being invalidate because project ef5f14d7e8084ef68b9bb6820272e787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:05.280414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:05.281648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-64b1d492-ccd5-4e1a-913c-dfb0cf608482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:05.284662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 882/1776] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:04 2022] DELETE /identity/v3/projects/ef5f14d7e8084ef68b9bb6820272e787 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:05.292677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.293618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.296886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.324521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:05.326574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:05.445561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 895/1777] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:05.463988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b9405-df2d-4cfa-bf78-22963da90078 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.472842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b9405-df2d-4cfa-bf78-22963da90078 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.487573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b9405-df2d-4cfa-bf78-22963da90078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.531505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8e9b9405-df2d-4cfa-bf78-22963da90078 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:05.534391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8e9b9405-df2d-4cfa-bf78-22963da90078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:05.559905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 883/1778] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:05.565776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:05.593261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda8c5a-fc6a-460b-b5c1-acf10ceeedd9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:05.606920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:05.608633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.609654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.610441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.622733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:05.622733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:05.639311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1156a-bf4f-4daf-9dac-e6e68b03b711 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.641226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1156a-bf4f-4daf-9dac-e6e68b03b711 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.641732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1156a-bf4f-4daf-9dac-e6e68b03b711 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.642902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:05.644965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 884/1779] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:27:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:27:05.685262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 896/1780] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:05.691797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1156a-bf4f-4daf-9dac-e6e68b03b711 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:05.718595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:05.723981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:05.731912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1156a-bf4f-4daf-9dac-e6e68b03b711 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:05.741080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24df738-5539-4f9a-8747-54b0468ac524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:05.742735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24df738-5539-4f9a-8747-54b0468ac524 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.748159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24df738-5539-4f9a-8747-54b0468ac524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.749022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24df738-5539-4f9a-8747-54b0468ac524 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.750294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24df738-5539-4f9a-8747-54b0468ac524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:05.757060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24df738-5539-4f9a-8747-54b0468ac524 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:05.758900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eae3b-6b74-4dec-a830-05f63b57cf92 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:05.777277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:05.779873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.780460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.780981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] PATH_INFO: `/v3/users/1c21ce575ee240a6b8dba5dfa36ab797` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.784940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c21ce575ee240a6b8dba5dfa36ab797)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:05.793463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:05.893452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 885/1781] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:05 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:05.899019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b17a-b071-4092-9283-53860199f598 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:05.899532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b17a-b071-4092-9283-53860199f598 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:05.900023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b17a-b071-4092-9283-53860199f598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:05.932456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5069b17a-b071-4092-9283-53860199f598 None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:05.933781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5069b17a-b071-4092-9283-53860199f598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:06.009385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 886/1782] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:27:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:06.010341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b17a-b071-4092-9283-53860199f598 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:06.107636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b17a-b071-4092-9283-53860199f598 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:06.116107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba3e6-4509-4928-85d7-926e8d48d567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:06.117374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba3e6-4509-4928-85d7-926e8d48d567 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.118101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba3e6-4509-4928-85d7-926e8d48d567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.118499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba3e6-4509-4928-85d7-926e8d48d567 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.119757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ba3e6-4509-4928-85d7-926e8d48d567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:06.124032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ba3e6-4509-4928-85d7-926e8d48d567 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:06.132788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c21ce575ee240a6b8dba5dfa36ab797'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:06.147710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c21ce575ee240a6b8dba5dfa36ab797'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:06.148515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c21ce575ee240a6b8dba5dfa36ab797'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:06.163490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c21ce575ee240a6b8dba5dfa36ab797'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:06.164276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c21ce575ee240a6b8dba5dfa36ab797'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:06.309674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 897/1783] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:05 2022] DELETE /identity/v3/users/1c21ce575ee240a6b8dba5dfa36ab797 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:06.311522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:06.321312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:06.339604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 887/1784] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:06 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:06.345172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.346117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.346812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.390616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:06.392800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:06.537556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ba73d-62ad-4643-a1ce-7253cf3bfde0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:06.547639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:06.548368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.548659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.548956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.551264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:06.562821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 888/1785] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:06.562821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:06.563791 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:06.580809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:06.583693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:06.593352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974a786-4f14-4822-9c34-59bc6b221b74 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:06.612369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:06.613070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.613363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.616279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.616279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:06.616648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:06.627803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 898/1786] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:06.628996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:06.637857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:06.654085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 889/1787] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:06.655289 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:06.673768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a32d42-ef7b-42ce-aca9-7ebde0af42ee admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:06.684594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:06.685534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.685952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.686756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.688528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:06.696203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:06.726213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 890/1788] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:06.727895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:06.750760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e415a9-32e8-4ad1-ac19-2748d863340d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:06.759177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747f8cd-6786-4042-876c-4516c427b36f service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:06.764900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78eb4a-ce3b-4aae-80de-71baeaf94b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:06.766395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78eb4a-ce3b-4aae-80de-71baeaf94b69 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.766918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78eb4a-ce3b-4aae-80de-71baeaf94b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.767552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78eb4a-ce3b-4aae-80de-71baeaf94b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.776063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:06.781105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:06.782226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:06.783169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78eb4a-ce3b-4aae-80de-71baeaf94b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:06.783786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] PATH_INFO: `/v3/projects/5880448ed29c420a930d21d90651bddb/users/4913018d7e9d4834a83f20815cf32227/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:06.785722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78eb4a-ce3b-4aae-80de-71baeaf94b69 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:06.802380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 899/1789] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:06.820543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5880448ed29c420a930d21d90651bddb, user_id=4913018d7e9d4834a83f20815cf32227, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:06.826790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:06.990780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 891/1790] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:06 2022] PUT /identity/v3/projects/5880448ed29c420a930d21d90651bddb/users/4913018d7e9d4834a83f20815cf32227/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:07.014742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:07.015340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:07.016368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:07.125998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] MFA Rules not processed for user `4913018d7e9d4834a83f20815cf32227`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:07.129301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:07.426208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 900/1791] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:07.431100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:07.524722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30877631-53b5-4151-83f8-224e14dc9693 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:07.530727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655f995-da55-4508-994d-328ca80ad271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:07.531488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655f995-da55-4508-994d-328ca80ad271 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:07.531714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655f995-da55-4508-994d-328ca80ad271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:07.531994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655f995-da55-4508-994d-328ca80ad271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:07.532982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655f995-da55-4508-994d-328ca80ad271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:07.535235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655f995-da55-4508-994d-328ca80ad271 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:07.697594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 892/1792] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:07 2022] POST /identity/v3/projects => generated 385 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:07.698980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:07.714355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348c4bd-2808-49c8-8c81-f58f1eee3831 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:07.795196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172447d-d831-4420-8668-edccac0525c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:07.796041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172447d-d831-4420-8668-edccac0525c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:07.796430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172447d-d831-4420-8668-edccac0525c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:07.796830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172447d-d831-4420-8668-edccac0525c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:07.797679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172447d-d831-4420-8668-edccac0525c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:07.800089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172447d-d831-4420-8668-edccac0525c9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:07.930714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e17ba-ef61-4578-8d03-02743508fe69 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:07.930887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e17ba-ef61-4578-8d03-02743508fe69 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:07.931353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e17ba-ef61-4578-8d03-02743508fe69 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:07.934956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 893/1793] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:27:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:27:07.951004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 901/1794] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:07 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:07.959187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e17ba-ef61-4578-8d03-02743508fe69 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:07.971336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:07.971509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:07.971867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:07.983666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e17ba-ef61-4578-8d03-02743508fe69 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:08.013240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:08.014477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:08.015076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:08.015583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:08.017021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:08.020201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:08.024444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] MFA Rules not processed for user `fa5724e78cc848a4af84a78bac03f64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:08.029895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:08.048666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 894/1795] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:08.051012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:08.075491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470299b6-3351-4e74-b62e-efe9344d52f3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:08.086520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:08.088985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:08.089861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:08.090704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:08.092533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:08.096691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:08.125497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 895/1796] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:08.139830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:08.164787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eeb6c1-7362-4734-8265-9472bf66ca8f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:08.176594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 902/1797] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:27:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:08.177511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f4ad-54da-4355-a638-3da805832cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:08.179489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f4ad-54da-4355-a638-3da805832cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:08.180024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f4ad-54da-4355-a638-3da805832cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:08.180585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f4ad-54da-4355-a638-3da805832cfd admin admin] PATH_INFO: `/v3/projects/2cbaec88e15d49cfb09dbcc9f89b17c5/users/a037a4b140124502939acecfd78a6b0b/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:08.200904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:08.214623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f4ad-54da-4355-a638-3da805832cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbaec88e15d49cfb09dbcc9f89b17c5, user_id=a037a4b140124502939acecfd78a6b0b, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:08.215173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5bcd1-6ab0-4a1d-8aa5-1a5708595933 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:08.222251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:08.222697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:08.223175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:08.223351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f4ad-54da-4355-a638-3da805832cfd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:08.223510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:08.236074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:08.238335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:08.256572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 903/1798] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:08.257970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:08.260344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:08.288253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab8ee-c57e-4741-b7b6-b05a605f13e4 service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:08.304263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:08.305245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:08.305525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:08.305800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] PATH_INFO: `/v3/users/00f0e14454144adbb37c4799f35c6415` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:08.307569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f0e14454144adbb37c4799f35c6415)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:08.310146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:08.442488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 896/1799] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:08 2022] PUT /identity/v3/projects/2cbaec88e15d49cfb09dbcc9f89b17c5/users/a037a4b140124502939acecfd78a6b0b/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:08.466609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:08.467135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:08.470885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:08.546175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] MFA Rules not processed for user `a037a4b140124502939acecfd78a6b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:08.551012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:08.563978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:08.640137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f0e14454144adbb37c4799f35c6415'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:08.683146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f0e14454144adbb37c4799f35c6415'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:08.688669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f0e14454144adbb37c4799f35c6415'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:08.729491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f0e14454144adbb37c4799f35c6415'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:08.731736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f0e14454144adbb37c4799f35c6415'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:08.785291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 897/1800] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:08.786389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:08.787856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:08.880728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 904/1801] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:08 2022] DELETE /identity/v3/users/00f0e14454144adbb37c4799f35c6415 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:08.882164 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:08.888904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:08.900794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5ccf2-2635-4623-ac96-d9e65f1422b1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:09.026115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:09.031462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:09.031462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:09.031462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:09.031462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:09.033963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:09.063158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ee42-1045-40f5-b901-27226d4dec0c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:09.068035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:09.069683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:09.072627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:09.072932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] PATH_INFO: `/v3/projects/edacca4533664b77bc9a033e11961002` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:09.076390 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=edacca4533664b77bc9a033e11961002)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:09.078345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:09.183231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 898/1802] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:08 2022] POST /identity/v3/projects => generated 385 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:09.187378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:09.306349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edacca4533664b77bc9a033e11961002', 'request_id': 'req-db586f55-3a6b-4ef5-afd2-fbb75501a57d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.324119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edacca4533664b77bc9a033e11961002', 'request_id': 'req-db586f55-3a6b-4ef5-afd2-fbb75501a57d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.345548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6cfe8-b059-4ed3-919f-929a35f84a6a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:09.440550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edacca4533664b77bc9a033e11961002', 'request_id': 'req-db586f55-3a6b-4ef5-afd2-fbb75501a57d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.459058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] The token cache is being invalidate because project edacca4533664b77bc9a033e11961002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:09.459593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.460888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.462658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 905/1803] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:08 2022] DELETE /identity/v3/projects/edacca4533664b77bc9a033e11961002 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:09.463469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:09.465901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:09.494027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:09.495364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:09.495839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:09.497874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:09.498827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:09.516938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:09.604462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db586f55-3a6b-4ef5-afd2-fbb75501a57d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:09.631270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:09.631270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:09.631270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:09.631270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] PATH_INFO: `/v3/projects/4cd397ed9d7b4db9b1a818842803d8df` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:09.631822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd397ed9d7b4db9b1a818842803d8df)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:09.631822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:09.753614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 899/1804] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:09 2022] POST /identity/v3/users => generated 299 bytes in 567 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:09.754499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:09.756202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:09.769484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0b33-02a0-4181-8df7-22fce679d1b2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:09.775472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:09.776724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:09.777006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:09.777412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] PATH_INFO: `/v3/users/a8bc95157be2461b82fb23492fd5816e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:09.779856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bc95157be2461b82fb23492fd5816e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:09.782399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:09.815416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:09.861606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd397ed9d7b4db9b1a818842803d8df', 'request_id': 'req-6852c748-559a-4dff-997f-90aac4afef43'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.868526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd397ed9d7b4db9b1a818842803d8df', 'request_id': 'req-6852c748-559a-4dff-997f-90aac4afef43'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.955367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd397ed9d7b4db9b1a818842803d8df', 'request_id': 'req-6852c748-559a-4dff-997f-90aac4afef43'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.969174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] The token cache is being invalidate because project 4cd397ed9d7b4db9b1a818842803d8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:09.969598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.970396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:09.972099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 906/1805] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:09 2022] DELETE /identity/v3/projects/4cd397ed9d7b4db9b1a818842803d8df => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:09.972973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:09.974413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:10.049027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852c748-559a-4dff-997f-90aac4afef43 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:10.053845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:10.054535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:10.054812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:10.055085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:10.055822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:10.057471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:10.069994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 907/1806] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:10.070897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:10.073078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:10.088530 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b811fd4-90bd-4611-b185-ea6888a8e07f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:10.093711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:10.094600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:10.094884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:10.095160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] PATH_INFO: `/v3/users/679aa395b2a1436a91d98291916ee722` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:10.096024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bc95157be2461b82fb23492fd5816e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.096555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=679aa395b2a1436a91d98291916ee722)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:10.097062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bc95157be2461b82fb23492fd5816e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.099007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:10.105622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bc95157be2461b82fb23492fd5816e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.122895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bc95157be2461b82fb23492fd5816e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.123760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bc95157be2461b82fb23492fd5816e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.463902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 900/1807] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:09 2022] DELETE /identity/v3/users/a8bc95157be2461b82fb23492fd5816e => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:10.469815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:10.472079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:10.565612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '679aa395b2a1436a91d98291916ee722'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.579566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3e50-8120-4a1a-b660-ed5623e92869 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:10.586655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '679aa395b2a1436a91d98291916ee722'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.588677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '679aa395b2a1436a91d98291916ee722'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.614880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:10.622838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:10.623739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:10.626493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '679aa395b2a1436a91d98291916ee722'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.627267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] PATH_INFO: `/v3/projects/6c00221de6ce46208c5cc6e7f3dba889/users/e6ddd09e20524c57b25671a8f6238c7d/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:10.627444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '679aa395b2a1436a91d98291916ee722'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:10.664069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c00221de6ce46208c5cc6e7f3dba889, user_id=e6ddd09e20524c57b25671a8f6238c7d, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:10.674834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:10.735545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 908/1808] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:10 2022] DELETE /identity/v3/users/679aa395b2a1436a91d98291916ee722 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:10.784962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 901/1809] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:10 2022] PUT /identity/v3/projects/6c00221de6ce46208c5cc6e7f3dba889/users/e6ddd09e20524c57b25671a8f6238c7d/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:10.790510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:10.795962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:10.933012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae9b62-988f-4f35-bffa-08141c3b5d53 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:11.044739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc188b2-aec5-4653-86f4-ce1e3d540a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:11.045934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc188b2-aec5-4653-86f4-ce1e3d540a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.046440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc188b2-aec5-4653-86f4-ce1e3d540a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.046953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc188b2-aec5-4653-86f4-ce1e3d540a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.048366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc188b2-aec5-4653-86f4-ce1e3d540a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:11.051571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc188b2-aec5-4653-86f4-ce1e3d540a50 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:11.072346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 909/1810] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:11.078484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:11.096636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec239e-b603-4d00-a55f-a65682f4c978 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:11.103306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:11.104465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.104924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.105325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] PATH_INFO: `/v3/projects/6c00221de6ce46208c5cc6e7f3dba889/users/e6ddd09e20524c57b25671a8f6238c7d/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.108072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c00221de6ce46208c5cc6e7f3dba889, user_id=e6ddd09e20524c57b25671a8f6238c7d, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:11.111810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:11.129074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 902/1811] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:11 2022] PUT /identity/v3/projects/6c00221de6ce46208c5cc6e7f3dba889/users/e6ddd09e20524c57b25671a8f6238c7d/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:11.144668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623c176-f112-4134-bf5c-300296330065 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.145785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623c176-f112-4134-bf5c-300296330065 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.145785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623c176-f112-4134-bf5c-300296330065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.161744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:11.165546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:11.205580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0623c176-f112-4134-bf5c-300296330065 None None] MFA Rules not processed for user `e6ddd09e20524c57b25671a8f6238c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:11.207473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0623c176-f112-4134-bf5c-300296330065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:11.337457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 910/1812] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:11.361866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623c176-f112-4134-bf5c-300296330065 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:11.368162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0623c176-f112-4134-bf5c-300296330065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:11.382156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfce867-5566-4e55-a810-acc103d40785 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:11.388564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:11.389465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.389812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.390060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] PATH_INFO: `/v3/projects/ac7132cb59ee46f481f7fae5063518bd` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.391379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7132cb59ee46f481f7fae5063518bd)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:11.393419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:11.468740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623c176-f112-4134-bf5c-300296330065 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:11.475653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:11.476953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.477357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.477938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.481172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:11.545648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:11.570682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7132cb59ee46f481f7fae5063518bd', 'request_id': 'req-4db731c8-743e-4966-9abe-65564d2d091f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:11.587092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7132cb59ee46f481f7fae5063518bd', 'request_id': 'req-4db731c8-743e-4966-9abe-65564d2d091f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:11.622883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:11.625613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efadb65-e2a6-41f5-9b76-95a1b887026f service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:11.647470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 911/1813] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:11.734467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7132cb59ee46f481f7fae5063518bd', 'request_id': 'req-4db731c8-743e-4966-9abe-65564d2d091f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:11.749926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] The token cache is being invalidate because project ac7132cb59ee46f481f7fae5063518bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:11.751988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:11.753553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:11.755521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 903/1814] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:11 2022] DELETE /identity/v3/projects/ac7132cb59ee46f481f7fae5063518bd => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:11.768314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf010c-6443-458a-9bfa-ca9051ec3d52 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.768877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf010c-6443-458a-9bfa-ca9051ec3d52 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.769392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf010c-6443-458a-9bfa-ca9051ec3d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.780073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:11.794453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:11.817795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bccf010c-6443-458a-9bfa-ca9051ec3d52 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:11.835371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bccf010c-6443-458a-9bfa-ca9051ec3d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:11.936629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db731c8-743e-4966-9abe-65564d2d091f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:11.942394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:11.943554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.943993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.944775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] PATH_INFO: `/v3/projects/a3cf9f1769d34933b2ad6929059c7478` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:11.946524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cf9f1769d34933b2ad6929059c7478)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:11.948642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:11.984777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 912/1815] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:11.997679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:11.998058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:11.998570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.033410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:12.035674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:12.049088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 913/1816] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:12.054476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.069132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf9862-8b58-48c1-908e-0baa3326d000 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.076772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.077786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.078216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.078633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.079830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:12.080311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.083044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.105169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 914/1817] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:12.111894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.129936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1501e-2cae-41a4-a570-ecb2487b64ed admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.148271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.149462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.149945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.150329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.175645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.175645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.176222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cf9f1769d34933b2ad6929059c7478', 'request_id': 'req-ab9e5638-9765-446e-a7e3-f79dce791a5a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:12.181871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cf9f1769d34933b2ad6929059c7478', 'request_id': 'req-ab9e5638-9765-446e-a7e3-f79dce791a5a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:12.270419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cf9f1769d34933b2ad6929059c7478', 'request_id': 'req-ab9e5638-9765-446e-a7e3-f79dce791a5a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:12.294721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] The token cache is being invalidate because project a3cf9f1769d34933b2ad6929059c7478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:12.295035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:12.295238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 915/1818] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/projects => generated 369 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:12.297220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.297454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ab9e5638-9765-446e-a7e3-f79dce791a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:12.300407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:12.306114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 904/1819] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:11 2022] DELETE /identity/v3/projects/a3cf9f1769d34933b2ad6929059c7478 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:12.308210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7815d-a148-4967-bc0b-2593c358ea6f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.309090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7815d-a148-4967-bc0b-2593c358ea6f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.309873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7815d-a148-4967-bc0b-2593c358ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.334908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f0a7815d-a148-4967-bc0b-2593c358ea6f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:12.337111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f0a7815d-a148-4967-bc0b-2593c358ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:12.421500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 905/1820] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:12.428360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.428695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.429004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.455766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:12.457558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:12.461034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161acd-5113-45a7-89ff-ac121b9d603b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.473443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.474838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 906/1821] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:12.481370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.487531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.487531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.487873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.488014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.488957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.513340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544828b-ac47-4722-bc44-cb21b5a78e5a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.522068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.523696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.523840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.524337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.525857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:12.526417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.530363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.584055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 907/1822] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:12.589110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.605918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 916/1823] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/users => generated 285 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:12.615200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.622764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c6bd-9b64-46cd-9c8b-129504b667b4 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.635531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.637525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.638020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.639105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.639934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.646498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.650007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb13d-ca3b-4ac7-87c9-913dfd0219d7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.668121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.669727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.670209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.670667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.671618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.677802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.701299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 917/1824] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:12.706763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.719150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c0e7d-613b-4d11-b050-b85a49a21cd5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.732092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.732092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.732092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.732092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.732952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.732952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.741942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 918/1825] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:12.747331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.756976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:12.762783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b8320-ab12-4921-9a7e-abfebc20a0ac admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.770032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d898a-c382-468e-870a-8c845cfcfad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:12.770969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d898a-c382-468e-870a-8c845cfcfad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.771251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d898a-c382-468e-870a-8c845cfcfad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.771544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d898a-c382-468e-870a-8c845cfcfad6 admin admin] PATH_INFO: `/v3/projects/1cc8662a17124f7b9d209896c46b1533/users/333ff0717c024b99b08a7eec8283a9e6/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:12.791089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d898a-c382-468e-870a-8c845cfcfad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc8662a17124f7b9d209896c46b1533, user_id=333ff0717c024b99b08a7eec8283a9e6, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:12.809378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d898a-c382-468e-870a-8c845cfcfad6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:12.834279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 908/1826] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/projects => generated 365 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:12.836508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:12.969693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9e3ee-f1ad-43b8-90e6-fa7b5df561a2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:12.973334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 919/1827] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:12 2022] PUT /identity/v3/projects/1cc8662a17124f7b9d209896c46b1533/users/333ff0717c024b99b08a7eec8283a9e6/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:12.974905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:12.975194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:12.976476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.065836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] MFA Rules not processed for user `333ff0717c024b99b08a7eec8283a9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:13.067167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:13.146835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.147815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.148118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.149106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.151729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.157894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.199335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 920/1828] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:13.217776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.230480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ed61b-f7f9-4ded-8c82-bae2e6e12d7a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.237285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.237853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.238131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.238405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.239146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.241151 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.399494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 909/1829] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:12 2022] POST /identity/v3/users => generated 283 bytes in 564 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:13.403870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 921/1830] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:13 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:13.405234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.408640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.416589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60384e57-c243-4305-9f64-bcfc1e1fe995 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.521603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77863bd1-350d-408c-847b-ce673b64bdc8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.525956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.528091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.528366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.528738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.529034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.529382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.529537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.529693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.530654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.530882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.532700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.534802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.559902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 910/1831] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:13.565105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.594455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51501be-8cf0-4f94-b9eb-6217564a49b7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.605368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.608724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.609339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.609806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.610809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.618076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.649674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 922/1832] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:13 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:13.656203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.658834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 911/1833] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:13.672852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab74dc-ed1e-41ae-b734-a14de64d0981 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.674920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.679733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.680069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.680792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.681085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.682069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.685345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.700636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 923/1834] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:13.709874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.713862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00ad1-d97e-4f84-8535-3d2bc2726dd7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.723524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e37e23-8f1e-4745-8f04-8d92cb307598 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:13.731127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.731792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.732082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.733226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] PATH_INFO: `/v3/projects/f8c23cf85ad64008afbaffde4ca9319a/users/610bf4bac3454acf8c6b27807b1acc97/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.743397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024623bd-7ec6-42e7-8274-c843875f86f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:13.747154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024623bd-7ec6-42e7-8274-c843875f86f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.749206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024623bd-7ec6-42e7-8274-c843875f86f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.750085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024623bd-7ec6-42e7-8274-c843875f86f7 admin admin] PATH_INFO: `/v3/projects/b2847d67a939437aa184093ef615e72d/users/fec8ce0cbaa14cd9bb6dc67a5515714f/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:13.760436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c23cf85ad64008afbaffde4ca9319a, user_id=610bf4bac3454acf8c6b27807b1acc97, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.767982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.804041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024623bd-7ec6-42e7-8274-c843875f86f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2847d67a939437aa184093ef615e72d, user_id=fec8ce0cbaa14cd9bb6dc67a5515714f, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:13.807564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024623bd-7ec6-42e7-8274-c843875f86f7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:13.873329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 924/1835] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:13 2022] PUT /identity/v3/projects/f8c23cf85ad64008afbaffde4ca9319a/users/610bf4bac3454acf8c6b27807b1acc97/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:13.875753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:13.908431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 912/1836] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:13 2022] PUT /identity/v3/projects/b2847d67a939437aa184093ef615e72d/users/fec8ce0cbaa14cd9bb6dc67a5515714f/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:13.919714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325838f-4380-4141-92fb-45e3413a008c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:13.920003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325838f-4380-4141-92fb-45e3413a008c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:13.920360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325838f-4380-4141-92fb-45e3413a008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.005365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5325838f-4380-4141-92fb-45e3413a008c None None] MFA Rules not processed for user `fec8ce0cbaa14cd9bb6dc67a5515714f`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:14.008766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5325838f-4380-4141-92fb-45e3413a008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:14.050470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42654c-d19f-4243-b61c-d907739547f8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.148452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 913/1837] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:14.153041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325838f-4380-4141-92fb-45e3413a008c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.164701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.164701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.164701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.164701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.165464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.165551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.180568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325838f-4380-4141-92fb-45e3413a008c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.192761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 925/1838] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:14.197896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.198709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.201832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.202158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.202945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.205769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.205974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.232023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77438822-e756-49ad-bda3-a709281b47f0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.271711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4552fa-837d-46b2-b173-5f13877b1e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.271711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4552fa-837d-46b2-b173-5f13877b1e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.271711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4552fa-837d-46b2-b173-5f13877b1e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.271711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4552fa-837d-46b2-b173-5f13877b1e7e admin admin] PATH_INFO: `/v3/projects/f8c23cf85ad64008afbaffde4ca9319a/users/610bf4bac3454acf8c6b27807b1acc97/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.273051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4552fa-837d-46b2-b173-5f13877b1e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c23cf85ad64008afbaffde4ca9319a, user_id=610bf4bac3454acf8c6b27807b1acc97, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.273540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4552fa-837d-46b2-b173-5f13877b1e7e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.286579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 926/1839] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:14 2022] PUT /identity/v3/projects/f8c23cf85ad64008afbaffde4ca9319a/users/610bf4bac3454acf8c6b27807b1acc97/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:14.301192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.301442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.301820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.338001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 914/1840] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:14 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:14.343672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.370158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] MFA Rules not processed for user `610bf4bac3454acf8c6b27807b1acc97`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:14.372763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:14.377267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:14.524357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86da2-c06e-4b6a-a05b-bcbfd3c5391a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.534864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.536197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.536768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.537337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.538764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.543478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.559964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 927/1841] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:14 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:14.585385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.590899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:14.641299 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 915/1842] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:14 2022] POST /identity/v3/users => generated 289 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:14.642991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.646978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:14.684059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc891c5-ffcb-4fa2-bd10-621e9e5f33df None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.689564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.689950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.690169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.690451 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.703428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.704961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.717961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 928/1843] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:27:14.718950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.736533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee38215-c963-4d2f-9772-640d1e6900ed service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.754867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aa263-ff7b-45da-986e-8d6988d30704 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.755325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.758011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.758011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.758011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.758361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.760709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.766444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.767374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.767961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.768467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.775357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:14.784525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 929/1844] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:14.788010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.799043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e43c5-a4aa-408f-9ae7-932968f39d94 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.809056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.812209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.812209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.812741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] PATH_INFO: `/v3/projects/25737c98539046a1b0420d2f52daa0a8/users/071b4440ac5b4be884ef6f2115b570f5/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.844587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25737c98539046a1b0420d2f52daa0a8, user_id=071b4440ac5b4be884ef6f2115b570f5, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.849418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.888432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:14.891542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:14.907682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 916/1845] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:14.908710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:14.920426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a7f2-d877-4b38-a1ba-e7bd4fae094e service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:14.926098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:14.926944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:14.927307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:14.927702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:14.942360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:14.984787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 930/1846] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:14 2022] PUT /identity/v3/projects/25737c98539046a1b0420d2f52daa0a8/users/071b4440ac5b4be884ef6f2115b570f5/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:14.991053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:15.082711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:15.087220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:15.125109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 917/1847] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:15.217179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2443-c5ae-4934-9d8d-57ef9e73740a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:15.224988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f203-bb52-4706-95f8-7da0a5e94bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:15.226802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f203-bb52-4706-95f8-7da0a5e94bee admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:15.227017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f203-bb52-4706-95f8-7da0a5e94bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:15.227688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f203-bb52-4706-95f8-7da0a5e94bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:15.230719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2f203-bb52-4706-95f8-7da0a5e94bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:15.233565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2f203-bb52-4706-95f8-7da0a5e94bee admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:15.265894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:15.282303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 931/1848] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:15.326759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dac3d4-56a2-44ef-994d-5a5c401eeedf service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:15.345754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:15.348052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:15.348629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:15.349153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] PATH_INFO: `/v3/projects/25737c98539046a1b0420d2f52daa0a8/users/071b4440ac5b4be884ef6f2115b570f5/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:15.354004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25737c98539046a1b0420d2f52daa0a8, user_id=071b4440ac5b4be884ef6f2115b570f5, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:15.361514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:15.382406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 918/1849] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:15 2022] PUT /identity/v3/projects/25737c98539046a1b0420d2f52daa0a8/users/071b4440ac5b4be884ef6f2115b570f5/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:15.400169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:15.400725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:15.401259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:15.465285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] MFA Rules not processed for user `071b4440ac5b4be884ef6f2115b570f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:15.479177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:15.607534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 932/1850] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:15.797135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:16.278276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:16.428977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:16.599533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40803-a4da-4d28-bb26-aa05b51f0446 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:16.612082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:16.613320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:16.614741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:16.615609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:16.640713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:16.650861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:16.681437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 919/1851] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:17.778247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:17.799722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfdfeb-2dd9-41ad-8ae3-02c63b2e7775 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:17.806069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:17.807701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:17.808470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:17.809118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:17.906919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:17.910385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:17.931726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 933/1852] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:18.719663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:18.736035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66d79b-9a59-4d3e-8fc3-202d9d8f8630 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:18.742713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:18.743775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:18.744280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:18.745401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:18.749293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:18.851417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:18.855078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93317a97-5ff7-4660-a5d5-630e977a0136 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:18.890765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 920/1853] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:20.398026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:21.028748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:22.561783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:22.685713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bfa0a-006c-4a5a-838a-5e6bf3a4be0e service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:22.693897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:22.695494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:22.695986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:22.696440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] PATH_INFO: `/v3/users/fec8ce0cbaa14cd9bb6dc67a5515714f` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:22.699650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec8ce0cbaa14cd9bb6dc67a5515714f)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:22.703521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:22.727566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:23.023539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec8ce0cbaa14cd9bb6dc67a5515714f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:23.048238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec8ce0cbaa14cd9bb6dc67a5515714f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:23.050233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec8ce0cbaa14cd9bb6dc67a5515714f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:23.080669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec8ce0cbaa14cd9bb6dc67a5515714f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:23.082518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec8ce0cbaa14cd9bb6dc67a5515714f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:23.199228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 934/1854] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:22 2022] DELETE /identity/v3/users/fec8ce0cbaa14cd9bb6dc67a5515714f => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:23.203364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:23.203647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:23.203916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:23.252914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:23.255084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:23.345207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 921/1855] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:23.392051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:23.394005 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:23.547339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c65a4-4707-418f-84a6-1e8234178291 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:23.561339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc31069-7121-43b0-8486-e5154c56ef63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:23.562425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc31069-7121-43b0-8486-e5154c56ef63 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:23.563417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc31069-7121-43b0-8486-e5154c56ef63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:23.563417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc31069-7121-43b0-8486-e5154c56ef63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:23.582823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc31069-7121-43b0-8486-e5154c56ef63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:23.584709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc31069-7121-43b0-8486-e5154c56ef63 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:23.602589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 935/1856] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:23.738441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:23.740699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:23.786535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751abdb8-aa45-4075-88f9-be4fca87e9b1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:23.811736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:23.814187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:23.814722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:23.815290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] PATH_INFO: `/v3/users/333ff0717c024b99b08a7eec8283a9e6` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:23.820828 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=333ff0717c024b99b08a7eec8283a9e6)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:23.826734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:24.099605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333ff0717c024b99b08a7eec8283a9e6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:24.101910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333ff0717c024b99b08a7eec8283a9e6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:24.120038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333ff0717c024b99b08a7eec8283a9e6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:24.145453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333ff0717c024b99b08a7eec8283a9e6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:24.146629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333ff0717c024b99b08a7eec8283a9e6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:24.255542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 922/1857] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:23 2022] DELETE /identity/v3/users/333ff0717c024b99b08a7eec8283a9e6 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:24.257459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:24.257751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:24.258041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:24.295118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:24.297603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:24.422228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 936/1858] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:24.441336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:24.444130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:24.586066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c66ef-05e0-41d5-a7c4-3bbe4f6a9c6f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:24.599744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb313a7-8d1c-4538-b200-82dfa618101f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:24.600999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb313a7-8d1c-4538-b200-82dfa618101f service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:24.601855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb313a7-8d1c-4538-b200-82dfa618101f service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:24.613165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb313a7-8d1c-4538-b200-82dfa618101f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:24.626408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb313a7-8d1c-4538-b200-82dfa618101f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:24.628272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb313a7-8d1c-4538-b200-82dfa618101f service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:24.631004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:24.633239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:24.644739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 923/1859] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:24.649271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253cbb9-870e-4479-9d35-9dfa9ff13541 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:24.667317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:24.671364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:24.671850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:24.672399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] PATH_INFO: `/v3/users/4913018d7e9d4834a83f20815cf32227` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:24.675361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4913018d7e9d4834a83f20815cf32227)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:24.680165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:24.812060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:25.048478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4913018d7e9d4834a83f20815cf32227'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:25.076749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4913018d7e9d4834a83f20815cf32227'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:25.080659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4913018d7e9d4834a83f20815cf32227'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:25.119249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4913018d7e9d4834a83f20815cf32227'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:25.119249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4913018d7e9d4834a83f20815cf32227'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:25.270115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6027232-e59f-45de-972e-2379be148f9a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:25.272730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6027232-e59f-45de-972e-2379be148f9a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:25.273573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6027232-e59f-45de-972e-2379be148f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:25.277104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 937/1860] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:24 2022] DELETE /identity/v3/users/4913018d7e9d4834a83f20815cf32227 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:25.322122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d6027232-e59f-45de-972e-2379be148f9a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:25.324097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d6027232-e59f-45de-972e-2379be148f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:25.351367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:25.355911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:25.568029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 924/1861] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:25.577490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6027232-e59f-45de-972e-2379be148f9a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:25.585019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d6027232-e59f-45de-972e-2379be148f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:25.608272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2d22-4546-4fc5-9218-50922953522a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:25.621197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:25.622168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:25.622462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:25.622748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] PATH_INFO: `/v3/projects/1cc8662a17124f7b9d209896c46b1533` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:25.625102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc8662a17124f7b9d209896c46b1533)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:25.627359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:25.676920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6027232-e59f-45de-972e-2379be148f9a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:25.683331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:25.684866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:25.685350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:25.685835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:25.727013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:25.727013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:25.727013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 925/1862] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:25.803619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:25.876952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc8662a17124f7b9d209896c46b1533', 'request_id': 'req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:25.891129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc8662a17124f7b9d209896c46b1533', 'request_id': 'req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.039049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:26.039431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:26.054645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf76c2-e412-4548-b34b-b6d2ac13de60 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:26.092673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc8662a17124f7b9d209896c46b1533', 'request_id': 'req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.137964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] The token cache is being invalidate because project 1cc8662a17124f7b9d209896c46b1533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:26.138516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.139683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.141849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 938/1863] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:25 2022] DELETE /identity/v3/projects/1cc8662a17124f7b9d209896c46b1533 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:26.146237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:26.147803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:26.148774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:26.150508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:26.150803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:26.151085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] PATH_INFO: `/v3/projects/b2847d67a939437aa184093ef615e72d` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:26.153775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2847d67a939437aa184093ef615e72d)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:26.157449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:26.167464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5c99-185c-4478-a3cc-d6c23fe3ac75 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:26.170543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:26.171576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:26.171937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:26.172339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] PATH_INFO: `/v3/users/610bf4bac3454acf8c6b27807b1acc97` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:26.173922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=610bf4bac3454acf8c6b27807b1acc97)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:26.175900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:26.396789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2847d67a939437aa184093ef615e72d', 'request_id': 'req-eeef4b0e-b919-4de3-acb1-72d85cef27f3'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.407100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2847d67a939437aa184093ef615e72d', 'request_id': 'req-eeef4b0e-b919-4de3-acb1-72d85cef27f3'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.573023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2847d67a939437aa184093ef615e72d', 'request_id': 'req-eeef4b0e-b919-4de3-acb1-72d85cef27f3'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.585721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] The token cache is being invalidate because project b2847d67a939437aa184093ef615e72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:26.586259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.587110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.589005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 926/1864] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:26 2022] DELETE /identity/v3/projects/b2847d67a939437aa184093ef615e72d => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:26.593209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:26.594821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:26.714560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4b0e-b919-4de3-acb1-72d85cef27f3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:26.725583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:26.727351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:26.727722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:26.728787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] PATH_INFO: `/v3/users/071b4440ac5b4be884ef6f2115b570f5` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:26.730576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=071b4440ac5b4be884ef6f2115b570f5)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:26.733153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:26.991264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '610bf4bac3454acf8c6b27807b1acc97'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.998437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '610bf4bac3454acf8c6b27807b1acc97'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:26.999606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '610bf4bac3454acf8c6b27807b1acc97'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.017367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '610bf4bac3454acf8c6b27807b1acc97'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.018370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '610bf4bac3454acf8c6b27807b1acc97'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.147816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 939/1865] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:26 2022] DELETE /identity/v3/users/610bf4bac3454acf8c6b27807b1acc97 => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:27.155147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:27.157784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:27.168693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071b4440ac5b4be884ef6f2115b570f5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.169915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071b4440ac5b4be884ef6f2115b570f5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.192874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071b4440ac5b4be884ef6f2115b570f5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.246384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071b4440ac5b4be884ef6f2115b570f5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.248690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071b4440ac5b4be884ef6f2115b570f5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.321421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e72a3a-a87f-42df-988a-a4315a989d11 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:27.336539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:27.338563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:27.339066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:27.340509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] PATH_INFO: `/v3/projects/5880448ed29c420a930d21d90651bddb` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:27.344391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5880448ed29c420a930d21d90651bddb)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:27.347347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:27.573140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 927/1866] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:26 2022] DELETE /identity/v3/users/071b4440ac5b4be884ef6f2115b570f5 => generated 0 bytes in 976 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:27.602601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5880448ed29c420a930d21d90651bddb', 'request_id': 'req-4ca1c663-1031-4f13-aa65-1417aeef5056'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.623334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5880448ed29c420a930d21d90651bddb', 'request_id': 'req-4ca1c663-1031-4f13-aa65-1417aeef5056'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.752984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5880448ed29c420a930d21d90651bddb', 'request_id': 'req-4ca1c663-1031-4f13-aa65-1417aeef5056'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.801102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] The token cache is being invalidate because project 5880448ed29c420a930d21d90651bddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:27.802058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.803247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:27.805670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 940/1867] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:27 2022] DELETE /identity/v3/projects/5880448ed29c420a930d21d90651bddb => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:27.812186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:27.816772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:28.100555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185a2f-92be-4f0e-a3e1-d3bdd00b5ae9 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:28.107412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:28.108421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:28.108758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:28.109077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] PATH_INFO: `/v3/users/a037a4b140124502939acecfd78a6b0b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:28.110480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a037a4b140124502939acecfd78a6b0b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:28.113700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:28.384171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:28.387648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:28.522457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a037a4b140124502939acecfd78a6b0b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.525230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a037a4b140124502939acecfd78a6b0b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.543092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a037a4b140124502939acecfd78a6b0b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.599792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a037a4b140124502939acecfd78a6b0b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.599792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a037a4b140124502939acecfd78a6b0b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.604943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c663-1031-4f13-aa65-1417aeef5056 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:28.619850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:28.621830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:28.622621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:28.623187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] PATH_INFO: `/v3/projects/f8c23cf85ad64008afbaffde4ca9319a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:28.628893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c23cf85ad64008afbaffde4ca9319a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:28.631122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:28.747515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 928/1868] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:27 2022] DELETE /identity/v3/users/a037a4b140124502939acecfd78a6b0b => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:28.750042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:28.756781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:28.791684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c23cf85ad64008afbaffde4ca9319a', 'request_id': 'req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.801064 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c23cf85ad64008afbaffde4ca9319a', 'request_id': 'req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.930070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c23cf85ad64008afbaffde4ca9319a', 'request_id': 'req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.953217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] The token cache is being invalidate because project f8c23cf85ad64008afbaffde4ca9319a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:28.953737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.955480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b2fd63b0-bcf2-47f7-af0a-fae1aba8aeb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:28.959857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 941/1869] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:28 2022] DELETE /identity/v3/projects/f8c23cf85ad64008afbaffde4ca9319a => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:28.968929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326f08-ea04-4faf-9a04-4c84c5eb3c82 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:28.969727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326f08-ea04-4faf-9a04-4c84c5eb3c82 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:28.974614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326f08-ea04-4faf-9a04-4c84c5eb3c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.033669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ac326f08-ea04-4faf-9a04-4c84c5eb3c82 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:29.036135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ac326f08-ea04-4faf-9a04-4c84c5eb3c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:29.041872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11bc54-18ce-4148-aaa0-aaffddba2df8 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:29.150574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:29.202676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 942/1870] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:29.210018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.210468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.210620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.239159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:29.240622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:29.251904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 943/1871] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:29.257207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:29.270978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:29.272018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.272353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.272737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] PATH_INFO: `/v3/projects/25737c98539046a1b0420d2f52daa0a8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.274141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25737c98539046a1b0420d2f52daa0a8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:29.276593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:29.283874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b2ec4-5d1b-4c7a-bcb0-10f03d51f45d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:29.289823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:29.291141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.291425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.295902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.295902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:29.295902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:29.295902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:29.309067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 944/1872] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:29.314211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:29.332701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b21cc-9bcd-4c85-a2a4-b1ff06ed5cc6 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:29.340664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:29.341887 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.342177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.342517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.343951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:29.345547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:29.475704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25737c98539046a1b0420d2f52daa0a8', 'request_id': 'req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:29.488933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25737c98539046a1b0420d2f52daa0a8', 'request_id': 'req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:29.540790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 945/1873] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:29 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:29.547030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:29.618975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25737c98539046a1b0420d2f52daa0a8', 'request_id': 'req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:29.658941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] The token cache is being invalidate because project 25737c98539046a1b0420d2f52daa0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:29.659441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:29.662865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:29.667704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 929/1874] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:28 2022] DELETE /identity/v3/projects/25737c98539046a1b0420d2f52daa0a8 => generated 0 bytes in 919 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:29.669676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:29.673209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:29.715407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b9f56-30ba-4c9d-83b4-07ca93c2cd01 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:29.721551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a7f9e-a71f-4b4c-93a0-9877e5897728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:29.722240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a7f9e-a71f-4b4c-93a0-9877e5897728 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.722527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a7f9e-a71f-4b4c-93a0-9877e5897728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.722806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a7f9e-a71f-4b4c-93a0-9877e5897728 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.723699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a7f9e-a71f-4b4c-93a0-9877e5897728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:29.725986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a7f9e-a71f-4b4c-93a0-9877e5897728 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:29.792817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f3fa7-0b2f-48cb-a57e-a560ae0e4851 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:29.802631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:29.810658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.810658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.810658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] PATH_INFO: `/v3/projects/2cbaec88e15d49cfb09dbcc9f89b17c5` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.810658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbaec88e15d49cfb09dbcc9f89b17c5)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:29.812094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:29.955216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 946/1875] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:29 2022] POST /identity/v3/users => generated 294 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:29.956762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:29.957048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:29.957327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:29.979143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:29.981672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:29.995552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 947/1876] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:29.996492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:29.997994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.010865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bb37e-54a1-45f4-9c38-7a4bd416d99c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.016210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86680f25-c66e-4020-aee2-6509af4427ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.016955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86680f25-c66e-4020-aee2-6509af4427ca admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.017238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86680f25-c66e-4020-aee2-6509af4427ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.017515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86680f25-c66e-4020-aee2-6509af4427ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.018287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86680f25-c66e-4020-aee2-6509af4427ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:30.019942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86680f25-c66e-4020-aee2-6509af4427ca admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:30.030382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbaec88e15d49cfb09dbcc9f89b17c5', 'request_id': 'req-538ff817-82f4-4887-b776-53635ecaad08'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.038869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 948/1877] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:30.038869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.038869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.038869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.040449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbaec88e15d49cfb09dbcc9f89b17c5', 'request_id': 'req-538ff817-82f4-4887-b776-53635ecaad08'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.060242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:30.061519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.139294 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 949/1878] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:30.140201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.150824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f27e1-36f8-4405-87c4-775442f59452 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.156135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.156875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.157610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.158181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.158397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbaec88e15d49cfb09dbcc9f89b17c5', 'request_id': 'req-538ff817-82f4-4887-b776-53635ecaad08'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.159244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:30.161162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:30.164691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:30.178584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 950/1879] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:30.178848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] The token cache is being invalidate because project 2cbaec88e15d49cfb09dbcc9f89b17c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:30.178848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.179118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.179251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.182757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 930/1880] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:29 2022] DELETE /identity/v3/projects/2cbaec88e15d49cfb09dbcc9f89b17c5 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:30.184328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.186733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.191642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c885ef3-e905-4425-8bb6-42bd0b91d7ab admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.196975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.197666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.197948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.198230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.199108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:30.199393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:30.201165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:30.212272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff817-82f4-4887-b776-53635ecaad08 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.216553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 951/1881] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:30.217711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.219578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.223286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.224213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.224532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.224812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] PATH_INFO: `/v3/projects/144eca287dc94b45a2dd6e40b8772c7b/users/afb5a341bf5e460abfc8df409c2a1d61/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.235777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf4e67-26bf-4139-86fc-ef5c6845a970 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.247418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.247418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.247418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.247418 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] PATH_INFO: `/v3/users/e6ddd09e20524c57b25671a8f6238c7d` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.247866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ddd09e20524c57b25671a8f6238c7d)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:30.247866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:30.253145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144eca287dc94b45a2dd6e40b8772c7b, user_id=afb5a341bf5e460abfc8df409c2a1d61, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:30.256611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:30.453969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 931/1882] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:30 2022] PUT /identity/v3/projects/144eca287dc94b45a2dd6e40b8772c7b/users/afb5a341bf5e460abfc8df409c2a1d61/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:30.454906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.456801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.567827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efc0e6-a95d-42cc-a8cc-cebf95e75af6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.573209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f96c13-cefb-41d8-a12a-3a13778de6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.574323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f96c13-cefb-41d8-a12a-3a13778de6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.575027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f96c13-cefb-41d8-a12a-3a13778de6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.575974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f96c13-cefb-41d8-a12a-3a13778de6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.576768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f96c13-cefb-41d8-a12a-3a13778de6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:30.580349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f96c13-cefb-41d8-a12a-3a13778de6f3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:30.609792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ddd09e20524c57b25671a8f6238c7d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.619304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ddd09e20524c57b25671a8f6238c7d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.620070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ddd09e20524c57b25671a8f6238c7d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.639068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ddd09e20524c57b25671a8f6238c7d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.639957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ddd09e20524c57b25671a8f6238c7d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:30.746993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 932/1883] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:30 2022] POST /identity/v3/projects => generated 381 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:30.746993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.746993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.746993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:30.788613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 952/1884] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:30 2022] DELETE /identity/v3/users/e6ddd09e20524c57b25671a8f6238c7d => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:30.789122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] MFA Rules not processed for user `afb5a341bf5e460abfc8df409c2a1d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:30.790543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.791758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.795544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.952584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 933/1885] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:30.957497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:30.959937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:30.991292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d687b-2eb9-43b0-ac1a-7ae8c2c727e0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:30.997311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:30.998536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:30.998865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:30.999143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.000208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.001931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.078417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f06642-8691-4473-a71c-ddf116c34431 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.087824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.088695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.089095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.089632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.090583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.093359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.165693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 953/1886] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:30 2022] POST /identity/v3/users => generated 291 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:31.173720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.181907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2f443-8193-4c5a-a947-1f48dca83d4a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.187436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.188131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.188458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.188772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.189483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.191502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.203659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 954/1887] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:31.208179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.224174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4acc5-33ec-4d0d-84ae-700ff9b1a6fd admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.231489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.232182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.232503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.232783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.233532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.235216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.248525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 955/1888] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:31.252556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.264907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44766cf-c546-4207-b84d-e858ff4aaa29 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.282585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6f86-4dcf-43d9-87b0-8d470cb744b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.283574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6f86-4dcf-43d9-87b0-8d470cb744b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.283857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6f86-4dcf-43d9-87b0-8d470cb744b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.284204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6f86-4dcf-43d9-87b0-8d470cb744b2 admin admin] PATH_INFO: `/v3/projects/b0d98390298349789c987890dfd5e655/users/5eef7aaa644b4c7a9debb9bc59903ad6/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.292802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 934/1889] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:30 2022] POST /identity/v3/projects => generated 387 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:31.294544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.313848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa6f86-4dcf-43d9-87b0-8d470cb744b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d98390298349789c987890dfd5e655, user_id=5eef7aaa644b4c7a9debb9bc59903ad6, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.317789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa6f86-4dcf-43d9-87b0-8d470cb744b2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.444524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b6581-f5bd-408f-a83c-af4d2dbca788 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.453669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.454931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.455727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.456228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.463244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.463244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.466871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 956/1890] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:31 2022] PUT /identity/v3/projects/b0d98390298349789c987890dfd5e655/users/5eef7aaa644b4c7a9debb9bc59903ad6/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:31.486415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.486701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.487007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.497634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:31.548382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] MFA Rules not processed for user `5eef7aaa644b4c7a9debb9bc59903ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:31.550625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:31.598659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 935/1891] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:31 2022] POST /identity/v3/users => generated 294 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:31.601087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.614172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e941b88-16c2-4005-941a-91669d7a95ee admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.669789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 957/1892] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:31.674662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.693538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.694260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.694547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.694824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.695595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.697882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.718010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 936/1893] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:31.718010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.732435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691a1ee-8cb3-49bd-9e62-a98ca8af0586 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.744952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.746187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.746711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.753503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.753503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.753503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.779516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 937/1894] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:31.779516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.782946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:31.786881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424ecab-75a0-4da8-aaf3-c26514147b00 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.793293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.794403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.794852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.795313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.796401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.798715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.808682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a36b8b-1668-45e9-bde8-11ff14b9dd71 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:31.817348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:31.818924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:31.819736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:31.820236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] PATH_INFO: `/v3/projects/6c00221de6ce46208c5cc6e7f3dba889` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:31.822711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c00221de6ce46208c5cc6e7f3dba889)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:31.826554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:31.912659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 958/1895] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:31 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:31.913679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:31.993825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c00221de6ce46208c5cc6e7f3dba889', 'request_id': 'req-e2d0d928-e233-47ba-a853-f1756ae5a828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:31.995480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787fa4-064d-4195-a1e5-583acfc646a6 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.003794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c00221de6ce46208c5cc6e7f3dba889', 'request_id': 'req-e2d0d928-e233-47ba-a853-f1756ae5a828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.083091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.084063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.084347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.084633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] PATH_INFO: `/v3/projects/77cd77b505b54ee4873b1a3b5caac059/users/112a8b82cd504dde829c0c880c268a1d/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.101526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cd77b505b54ee4873b1a3b5caac059, user_id=112a8b82cd504dde829c0c880c268a1d, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.104320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:32.114547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c00221de6ce46208c5cc6e7f3dba889', 'request_id': 'req-e2d0d928-e233-47ba-a853-f1756ae5a828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.132177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] The token cache is being invalidate because project 6c00221de6ce46208c5cc6e7f3dba889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:32.132741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.134101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.135945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 938/1896] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:31 2022] DELETE /identity/v3/projects/6c00221de6ce46208c5cc6e7f3dba889 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:32.137085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.138653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.156988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d928-e233-47ba-a853-f1756ae5a828 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.162214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eccff5-561e-4fd3-9e16-2e7deb2d0573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.162947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eccff5-561e-4fd3-9e16-2e7deb2d0573 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.163235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eccff5-561e-4fd3-9e16-2e7deb2d0573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.170576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eccff5-561e-4fd3-9e16-2e7deb2d0573 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.170576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eccff5-561e-4fd3-9e16-2e7deb2d0573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.170576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eccff5-561e-4fd3-9e16-2e7deb2d0573 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:32.239242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 959/1897] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:31 2022] PUT /identity/v3/projects/77cd77b505b54ee4873b1a3b5caac059/users/112a8b82cd504dde829c0c880c268a1d/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:32.240810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.243580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.278065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 939/1898] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:32 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:32.279776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389c25b-1312-45ac-99ea-be923e2e46ad None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.280317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389c25b-1312-45ac-99ea-be923e2e46ad None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.280830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389c25b-1312-45ac-99ea-be923e2e46ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.342090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946c3aa-0896-4aff-b8b5-ed505732cfcf admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.348160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.349582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.350039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.350490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] PATH_INFO: `/v3/users/c1a0478618b1430b9c8c609618257eca` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.350747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-4389c25b-1312-45ac-99ea-be923e2e46ad None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:32.353119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a0478618b1430b9c8c609618257eca)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.353291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4389c25b-1312-45ac-99ea-be923e2e46ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.356670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:32.366610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 940/1899] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:32.369500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.369983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.370444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.410726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] MFA Rules not processed for user `112a8b82cd504dde829c0c880c268a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:32.412441 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.521119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 941/1900] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:32.522382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.535524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8679b-b53d-44bd-b1e4-05af47685620 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.541979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94569894-2553-4876-a6b9-5f15b03edc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.542697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94569894-2553-4876-a6b9-5f15b03edc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.542988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94569894-2553-4876-a6b9-5f15b03edc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.543266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94569894-2553-4876-a6b9-5f15b03edc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.544018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94569894-2553-4876-a6b9-5f15b03edc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.545746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94569894-2553-4876-a6b9-5f15b03edc59 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:32.561633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 942/1901] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:32.561633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.561633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.561633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.582984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:32.584325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.599703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 943/1902] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:32.600629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.603326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.611783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a0478618b1430b9c8c609618257eca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.616146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649163a7-2087-429b-9806-dd4cb9707813 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.621420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.622086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.622372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.622905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.624838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.625629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:32.628884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a0478618b1430b9c8c609618257eca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.630327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a0478618b1430b9c8c609618257eca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.653221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a0478618b1430b9c8c609618257eca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.654441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a0478618b1430b9c8c609618257eca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:32.755688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 944/1903] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:32 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:32.755688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.755688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.768042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 960/1904] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:32 2022] DELETE /identity/v3/users/c1a0478618b1430b9c8c609618257eca => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:32.770411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.772830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.856755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ad28-7b04-47be-8d0d-02e8606d8c27 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.903720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f62eb5-9673-4daf-bc61-b539d9896abc admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.910536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.912640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.913254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.913733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.916088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:32.916668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.919423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:32.947042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 961/1905] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:32.950183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:32.954636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:32.978188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287ac1-7a20-4d28-b3d4-dfbbcd93106d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:32.983703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.983703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.983703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.983703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] PATH_INFO: `/v3/projects/8f5ad8809e144bfea78468e44f1f79e5/users/85cbda516c8344a0a5b27015d0ab0ffa/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.988582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:32.989782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:32.990332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:32.990879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:32.992328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:32.997125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.011153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5ad8809e144bfea78468e44f1f79e5, user_id=85cbda516c8344a0a5b27015d0ab0ffa, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.015758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.020611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:33.127541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 962/1906] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:32 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:33.128472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.141358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155fb56-e3f2-4bff-a90a-f34d44cccc22 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.147672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.148503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.149440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.149773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.150562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.152732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.168231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 945/1907] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:32 2022] PUT /identity/v3/projects/8f5ad8809e144bfea78468e44f1f79e5/users/85cbda516c8344a0a5b27015d0ab0ffa/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:33.169867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.256794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916253f-ace7-4ffe-998c-01aed90f7bc1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.263435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.266032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.266032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.266032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.266032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.267975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.280217 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 946/1908] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:33.281790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.296515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a663d-88d3-4488-8dbb-adc9e70d38a5 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.303960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.305489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.305962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.306426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.307676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.313632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 963/1909] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:33 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:33.313632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.314048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.314190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:33.327549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 947/1910] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:33.328858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.342947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a8e6-34f0-4b82-ac5c-8b47439fc25c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.403696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab8ad8-5af3-42e8-9a99-f3fe59d581c6 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.409381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.410400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.410750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.411032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] PATH_INFO: `/v3/projects/29bb9979415948f685ad1fe8d58bfb20` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.412441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29bb9979415948f685ad1fe8d58bfb20)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.414911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.431168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.432484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.432849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.433212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] PATH_INFO: `/v3/projects/484042f0149d4518bb5b4005626ccc24/users/7ccd80acfcbe402889b79e903887388a/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.452772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484042f0149d4518bb5b4005626ccc24, user_id=7ccd80acfcbe402889b79e903887388a, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.455724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.622249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 948/1911] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:33 2022] PUT /identity/v3/projects/484042f0149d4518bb5b4005626ccc24/users/7ccd80acfcbe402889b79e903887388a/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:33.623921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.638102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29bb9979415948f685ad1fe8d58bfb20', 'request_id': 'req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:33.652034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29bb9979415948f685ad1fe8d58bfb20', 'request_id': 'req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:33.715388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50af9e-dbe4-4da8-980e-a043369eb230 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.758146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29bb9979415948f685ad1fe8d58bfb20', 'request_id': 'req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:33.778581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] The token cache is being invalidate because project 29bb9979415948f685ad1fe8d58bfb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:33.779386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:33.780191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:33.782981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 964/1912] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:33 2022] DELETE /identity/v3/projects/29bb9979415948f685ad1fe8d58bfb20 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:33.784345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.786811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:33.816316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a700da3-c8ea-4283-a22e-1c4e307d2fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.818196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700da3-c8ea-4283-a22e-1c4e307d2fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.818698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700da3-c8ea-4283-a22e-1c4e307d2fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.819672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700da3-c8ea-4283-a22e-1c4e307d2fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.820702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a700da3-c8ea-4283-a22e-1c4e307d2fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.825916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a700da3-c8ea-4283-a22e-1c4e307d2fd7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.888537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb232f65-9a89-4fa4-b66c-20cf1d7f2a31 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:33.897084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:33.898721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.899266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.899877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] PATH_INFO: `/v3/projects/8f5ad8809e144bfea78468e44f1f79e5/users/85cbda516c8344a0a5b27015d0ab0ffa/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:33.903681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5ad8809e144bfea78468e44f1f79e5, user_id=85cbda516c8344a0a5b27015d0ab0ffa, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:33.909513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:33.929088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 965/1913] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:33 2022] PUT /identity/v3/projects/8f5ad8809e144bfea78468e44f1f79e5/users/85cbda516c8344a0a5b27015d0ab0ffa/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:33.931544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:33.933952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:33.975464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 949/1914] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:33 2022] POST /identity/v3/users => generated 293 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:33.975464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:33.975464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:33.975464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.000531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:34.002635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.021939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513b41b-4f72-4da4-ab55-17ec3a0d213d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.030283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011f774-096b-4be1-a5b2-eb3b3f1297f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.043728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011f774-096b-4be1-a5b2-eb3b3f1297f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.043728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011f774-096b-4be1-a5b2-eb3b3f1297f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.043728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011f774-096b-4be1-a5b2-eb3b3f1297f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.043728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011f774-096b-4be1-a5b2-eb3b3f1297f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.043728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011f774-096b-4be1-a5b2-eb3b3f1297f3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.047424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 966/1915] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:34.048933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.049384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.049755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.082669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] MFA Rules not processed for user `85cbda516c8344a0a5b27015d0ab0ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:34.084874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.104754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 950/1916] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:34.106411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.108696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.125884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e64f-2ef7-43f0-b2c2-bf690ea9efb2 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.131135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.131827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.132109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.132422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.133211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.134906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.146446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 951/1917] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:34.147336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.161545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d0e13-a09e-4bb2-ac44-c4f2d837260c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.168626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dd491-e87d-4a2a-afa3-4e6261658371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.169579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd491-e87d-4a2a-afa3-4e6261658371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.169866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd491-e87d-4a2a-afa3-4e6261658371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.170144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd491-e87d-4a2a-afa3-4e6261658371 admin admin] PATH_INFO: `/v3/projects/484042f0149d4518bb5b4005626ccc24/users/7ccd80acfcbe402889b79e903887388a/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.172252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dd491-e87d-4a2a-afa3-4e6261658371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484042f0149d4518bb5b4005626ccc24, user_id=7ccd80acfcbe402889b79e903887388a, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.175221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dd491-e87d-4a2a-afa3-4e6261658371 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.201591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 952/1918] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:34 2022] PUT /identity/v3/projects/484042f0149d4518bb5b4005626ccc24/users/7ccd80acfcbe402889b79e903887388a/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:34.201591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef426e-1499-4959-902d-e48c026d7f56 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.201591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef426e-1499-4959-902d-e48c026d7f56 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.201591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef426e-1499-4959-902d-e48c026d7f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.202216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 967/1919] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:34.202216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.211938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-1bef426e-1499-4959-902d-e48c026d7f56 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:34.213274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1bef426e-1499-4959-902d-e48c026d7f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.215217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:34.223489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2eae8-d099-4653-8d4a-b8f3b56275fe None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.291389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 953/1920] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:34.292912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.293142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.293482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.323105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.324441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.324908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.325372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.326643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.329617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.329804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] MFA Rules not processed for user `7ccd80acfcbe402889b79e903887388a`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:34.330924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.349301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 968/1921] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:34.350797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.355315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.419332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 954/1922] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:34.420542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.435332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977767b8-0c3e-4ca6-ba24-8633638e5c4f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.443465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.445068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.446049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.446483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.447777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:34.448270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.450871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.468721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 955/1923] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:34.469800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.472032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2584ace-ab53-42f2-9857-91dd70f8d597 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.477984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.478751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.479102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.479455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.483970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf8af4-b6d2-49c9-b64d-65b6278cddf1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.489789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.490976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.491325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.491806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] PATH_INFO: `/v3/projects/e808152608374179b672add7ce45e20f/users/c3b8b65dadb74f50b1ba5464854df8d8/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.517485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e808152608374179b672add7ce45e20f, user_id=c3b8b65dadb74f50b1ba5464854df8d8, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.522320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.565430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.566917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.578767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 969/1924] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:34.579809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.581651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.595338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e74296-59f0-476c-ba98-0f1dc71ad74c service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.619604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 956/1925] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:34 2022] PUT /identity/v3/projects/e808152608374179b672add7ce45e20f/users/c3b8b65dadb74f50b1ba5464854df8d8/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:34.619604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.666434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57df6aa-16d8-483c-8240-d4c51eccd15a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.667525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57df6aa-16d8-483c-8240-d4c51eccd15a service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.667937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57df6aa-16d8-483c-8240-d4c51eccd15a service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.668663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57df6aa-16d8-483c-8240-d4c51eccd15a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.670815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:34.730231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120db6e-11a3-4f11-8296-415ad9d214dd admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:34.737884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:34.738250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.738783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.739000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.739731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.742493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.775248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57df6aa-16d8-483c-8240-d4c51eccd15a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:34.776878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57df6aa-16d8-483c-8240-d4c51eccd15a service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:34.788062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 970/1926] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:34.789533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:34.789819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:34.790098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:34.842592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] MFA Rules not processed for user `c3b8b65dadb74f50b1ba5464854df8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:34.844808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:34.897419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 957/1927] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:34 2022] POST /identity/v3/projects => generated 379 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:34.904483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:34.976202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 971/1928] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:34.992192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:35.009501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a71d9-cd87-466d-8095-f9222daddc39 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.024358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019077-5b74-4dbe-b2a4-bdcc9d0db84a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.031245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.032550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.033026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.033478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.034683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.038401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:35.090040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.090719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.091003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.091279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.172844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 958/1929] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:34 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:35.178703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:35.190219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.201429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:35.207110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247ac1d-a758-40a8-aa04-b5697158e861 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.210424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 972/1930] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:35.212658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.213741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.214137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.214513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.215406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.217960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:35.236700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 959/1931] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:35.239465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:35.255149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dce47-4c0b-4623-b18b-79fbedb3644d service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.261180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd64e6-679f-45cd-8c5c-546dc50cd6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.262186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd64e6-679f-45cd-8c5c-546dc50cd6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.262618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd64e6-679f-45cd-8c5c-546dc50cd6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.263346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd64e6-679f-45cd-8c5c-546dc50cd6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.264745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd64e6-679f-45cd-8c5c-546dc50cd6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.267830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd64e6-679f-45cd-8c5c-546dc50cd6d5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:35.290698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 973/1932] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:35.292267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:35.306472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9208d5-c2c0-4103-b8ae-d83d8b88516e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.313396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.314889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.315376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.315844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] PATH_INFO: `/v3/projects/cabeb8e0571c4dd0b3ffdd0bbf4fb9f4/users/59823bde2ed74526a78ea40b737b88e7/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.347496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabeb8e0571c4dd0b3ffdd0bbf4fb9f4, user_id=59823bde2ed74526a78ea40b737b88e7, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.353417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:35.453224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 960/1933] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:35 2022] PUT /identity/v3/projects/cabeb8e0571c4dd0b3ffdd0bbf4fb9f4/users/59823bde2ed74526a78ea40b737b88e7/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:35.465712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.466116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.466496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.503804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] MFA Rules not processed for user `59823bde2ed74526a78ea40b737b88e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:35.505696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:35.591910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 974/1934] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:35.598692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:35.690777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78af20-3441-40b1-936d-87f7ce45e4ca admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.691392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:35.698648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.699342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.699626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.699907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.700670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.704003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:35.839983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 961/1935] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:35 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:35.845668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:35.858810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b13fbe-1fa7-4c2b-8497-ec58a1bc8b68 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:35.969964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:35.971069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:35.971510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:35.972023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:35.973488 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:35.977575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.202977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.229263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 975/1936] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:35 2022] POST /identity/v3/users => generated 297 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:36.239566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.250266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa888e-4509-4524-a1b0-d05fba41afe7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.255849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.256699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.257053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.257401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.258784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.260649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.275987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 976/1937] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:36.277170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.288984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206cf2b-704f-4a1d-903a-c776c0781df0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.310397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.312072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.312683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.313186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] PATH_INFO: `/v3/projects/dd39021a4ea448aaa1c5dd787a3ea9eb/users/818876a8d1a942dc8b71ea13de8cb474/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.334642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6239e21-aab0-4517-87ee-82a86dab2f30 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.344626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85cbda516c8344a0a5b27015d0ab0ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5ad8809e144bfea78468e44f1f79e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.346383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.346832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.347335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.348861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:36.349428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.352700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd39021a4ea448aaa1c5dd787a3ea9eb, user_id=818876a8d1a942dc8b71ea13de8cb474, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.352827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.359833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.380103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 962/1938] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:36.386647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.399210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643858f3-7127-4142-abac-b82299c768c5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.405282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85cbda516c8344a0a5b27015d0ab0ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5ad8809e144bfea78468e44f1f79e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.406208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.406573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.406901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.408119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.412384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.525020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 977/1939] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:36 2022] PUT /identity/v3/projects/dd39021a4ea448aaa1c5dd787a3ea9eb/users/818876a8d1a942dc8b71ea13de8cb474/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:36.530940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.562169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 963/1940] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:36 2022] POST /identity/v3/projects => generated 354 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:36.568803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.633994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e9c8-6d59-405d-b5d1-edec60f09760 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.672030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed0ec-37d0-434c-a75a-53033693edaf tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.680367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85cbda516c8344a0a5b27015d0ab0ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5ad8809e144bfea78468e44f1f79e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.681983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.682488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.682841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.683872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:36.684564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.687477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.703861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 964/1941] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:36.727699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.729215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.729771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.730198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.731211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.733419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.747999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 978/1942] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:36.752661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.756831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.765919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18586751-1e92-472c-926d-78ba224379e5 tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.772738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2676212-20f7-4887-bffb-92043ce64c6c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:36.773084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4adc6e-645d-4755-8ee6-83e2499b7f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.778349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4adc6e-645d-4755-8ee6-83e2499b7f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.778349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4adc6e-645d-4755-8ee6-83e2499b7f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.778349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4adc6e-645d-4755-8ee6-83e2499b7f57 admin admin] PATH_INFO: `/v3/projects/dd39021a4ea448aaa1c5dd787a3ea9eb/users/818876a8d1a942dc8b71ea13de8cb474/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.778349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4adc6e-645d-4755-8ee6-83e2499b7f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd39021a4ea448aaa1c5dd787a3ea9eb, user_id=818876a8d1a942dc8b71ea13de8cb474, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.781170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4adc6e-645d-4755-8ee6-83e2499b7f57 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.781669 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85cbda516c8344a0a5b27015d0ab0ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5ad8809e144bfea78468e44f1f79e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:36.783176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.783780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.784240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] PATH_INFO: `/v3/projects/2e36266ca30644d5a41fc44876a979e3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.786627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2e36266ca30644d5a41fc44876a979e3)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:36.790780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:36.797278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 965/1943] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:36 2022] PUT /identity/v3/projects/dd39021a4ea448aaa1c5dd787a3ea9eb/users/818876a8d1a942dc8b71ea13de8cb474/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:36.806142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:36.810137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:36.817245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:36.817245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:36.849371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] MFA Rules not processed for user `818876a8d1a942dc8b71ea13de8cb474`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:36.850649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:36.934976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 966/1944] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:36.948361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:36.963562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e36266ca30644d5a41fc44876a979e3', 'request_id': 'req-903eda8a-91fd-40ce-ab42-302776c7311f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:36.973155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e36266ca30644d5a41fc44876a979e3', 'request_id': 'req-903eda8a-91fd-40ce-ab42-302776c7311f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:37.020130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cf720-aa29-467a-b2ad-d521a1f2a210 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:37.079446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e36266ca30644d5a41fc44876a979e3', 'request_id': 'req-903eda8a-91fd-40ce-ab42-302776c7311f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:37.092703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:37.093853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:37.094418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:37.094925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:37.095832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] The token cache is being invalidate because project 2e36266ca30644d5a41fc44876a979e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:37.096225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:37.097202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:37.099436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 979/1945] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:36 2022] DELETE /identity/v3/projects/2e36266ca30644d5a41fc44876a979e3 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:37.175481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:37.177081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:37.189782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 967/1946] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:27:37.729343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:37.730833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:37.745354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eda8a-91fd-40ce-ab42-302776c7311f tempest-VolumeQuotaClassesTest-1342153346 tempest-VolumeQuotaClassesTest-1342153346-project-admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:37.751414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:37.752626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:37.753101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:37.753599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:37.757143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:37.862434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:37.863920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432969f0-6fca-4488-af39-0cbc54a607f6 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:37.875630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 980/1947] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:38.466427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:38.547995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:38.585830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:38.684558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b1b1-8e8f-4f24-aab0-5ff50d7743b9 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:38.693530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:38.696178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:38.696710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:38.697220 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] PATH_INFO: `/v3/users/5eef7aaa644b4c7a9debb9bc59903ad6` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:38.699574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eef7aaa644b4c7a9debb9bc59903ad6)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:38.703537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:39.030957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eef7aaa644b4c7a9debb9bc59903ad6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.032367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eef7aaa644b4c7a9debb9bc59903ad6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.046836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eef7aaa644b4c7a9debb9bc59903ad6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.068587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eef7aaa644b4c7a9debb9bc59903ad6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.069618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eef7aaa644b4c7a9debb9bc59903ad6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.223723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 968/1948] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:38 2022] DELETE /identity/v3/users/5eef7aaa644b4c7a9debb9bc59903ad6 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:39.228889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:39.229307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:39.229703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:39.249831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:39.251177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:39.262111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:39.336812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 981/1949] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:39.348034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:39.349766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:39.461825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0037fd3-d018-4429-8173-22be93b886c1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:39.465979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62634f3-d9be-49f0-8c53-c68d47767123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:39.466711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62634f3-d9be-49f0-8c53-c68d47767123 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:39.467065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62634f3-d9be-49f0-8c53-c68d47767123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:39.467566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62634f3-d9be-49f0-8c53-c68d47767123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:39.478547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62634f3-d9be-49f0-8c53-c68d47767123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:39.480051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62634f3-d9be-49f0-8c53-c68d47767123 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:39.496405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 969/1950] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:39.615203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:39.616958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:39.632844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a127b3-1457-4b82-8aa6-8baecba8b549 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:39.638848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:39.639701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:39.639982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:39.640927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] PATH_INFO: `/v3/users/c3b8b65dadb74f50b1ba5464854df8d8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:39.642391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b8b65dadb74f50b1ba5464854df8d8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:39.644490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:39.901212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b8b65dadb74f50b1ba5464854df8d8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.915163 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b8b65dadb74f50b1ba5464854df8d8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.917564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b8b65dadb74f50b1ba5464854df8d8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.946128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b8b65dadb74f50b1ba5464854df8d8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:39.947599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b8b65dadb74f50b1ba5464854df8d8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:40.038254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 982/1951] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:39 2022] DELETE /identity/v3/users/c3b8b65dadb74f50b1ba5464854df8d8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:40.057365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:40.057365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:40.057365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:40.078370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:40.080303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:40.179651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 970/1952] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:40.197097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:40.199638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:40.203292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:40.205813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:40.300384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:40.325677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5db67-fbcf-409d-a47c-8242564c16b1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:40.326672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78494153-a632-4174-8251-52fccf3b55f4 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:40.334347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:40.337790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:40.338298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:40.338861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:40.339380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:40.339489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:40.340007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:40.340267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:40.347577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:40.360193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:40.363058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:40.381814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 971/1953] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:27:40.383810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:40.386252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:40.413156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d9a31-6449-4649-a233-edd13a6bcaa9 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:40.421106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:40.422663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:40.423409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:40.423963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] PATH_INFO: `/v3/projects/b0d98390298349789c987890dfd5e655` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:40.426737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d98390298349789c987890dfd5e655)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:40.430280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:40.449761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:40.453112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:40.470500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 983/1954] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:40.584876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d98390298349789c987890dfd5e655', 'request_id': 'req-c24f8686-6e64-41fc-afd7-6793c517448e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:40.593003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d98390298349789c987890dfd5e655', 'request_id': 'req-c24f8686-6e64-41fc-afd7-6793c517448e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:40.719513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d98390298349789c987890dfd5e655', 'request_id': 'req-c24f8686-6e64-41fc-afd7-6793c517448e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:40.746309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] The token cache is being invalidate because project b0d98390298349789c987890dfd5e655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:40.747307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:40.748641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:40.752027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 972/1955] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:40 2022] DELETE /identity/v3/projects/b0d98390298349789c987890dfd5e655 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:40.756948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:40.779693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0562b8-1010-42ea-8043-be928c57d79c service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:40.883426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:40.884600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:40.888851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:40.888851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] PATH_INFO: `/v3/users/85cbda516c8344a0a5b27015d0ab0ffa` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:40.888851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=85cbda516c8344a0a5b27015d0ab0ffa)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:40.888851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:41.083794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:41.095599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:41.100395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:41.131677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f8686-6e64-41fc-afd7-6793c517448e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:41.137931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:41.138852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:41.139621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:41.139853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] PATH_INFO: `/v3/projects/e808152608374179b672add7ce45e20f` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:41.141166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e808152608374179b672add7ce45e20f)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:41.143364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:41.190040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85cbda516c8344a0a5b27015d0ab0ffa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.198363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85cbda516c8344a0a5b27015d0ab0ffa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.199194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85cbda516c8344a0a5b27015d0ab0ffa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.214204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85cbda516c8344a0a5b27015d0ab0ffa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.215063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-83b407ec-8a36-455d-8dc2-4b36de2331cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85cbda516c8344a0a5b27015d0ab0ffa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.312646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e808152608374179b672add7ce45e20f', 'request_id': 'req-28c5cb3f-d2d5-487a-8094-900f38da00c6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.321058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e808152608374179b672add7ce45e20f', 'request_id': 'req-28c5cb3f-d2d5-487a-8094-900f38da00c6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.374026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 984/1956] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:40 2022] DELETE /identity/v3/users/85cbda516c8344a0a5b27015d0ab0ffa => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:41.485406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e808152608374179b672add7ce45e20f', 'request_id': 'req-28c5cb3f-d2d5-487a-8094-900f38da00c6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.512245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] The token cache is being invalidate because project e808152608374179b672add7ce45e20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:41.513421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.514232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-28c5cb3f-d2d5-487a-8094-900f38da00c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:41.524474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 973/1957] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:41 2022] DELETE /identity/v3/projects/e808152608374179b672add7ce45e20f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:41.543029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:41.543572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:41.543967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:41.571040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:41.572741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:41.658040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 985/1958] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:41.673081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:41.673484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:41.674019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:41.708060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:41.710977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:41.726984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 974/1959] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:41.733303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:41.750355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c045f-040e-4aae-bcc0-5c941b7098e0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:41.762708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:41.769426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:41.769426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:41.769426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:41.769426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:41.769426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:41.772238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:41.798234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 986/1960] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:41.805342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:41.823032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e4cff-0c50-4d44-9d91-f208eb948b4c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:41.830725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:41.831417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:41.832552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:41.832552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:41.832966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:41.834779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:41.965448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 975/1961] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:41 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:41.971089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:41.993039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a52240-8f73-4d08-8e57-584525cd86af admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.022164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.027174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:42.092515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.093318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.093606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.093881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.094603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.096707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.150926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33927a5e-22d9-4094-9f71-ef6fb2a120ed admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.157670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.158581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.158867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.159146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] PATH_INFO: `/v3/projects/8f5ad8809e144bfea78468e44f1f79e5` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.160580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5ad8809e144bfea78468e44f1f79e5)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.162682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.197886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:42.205462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 987/1962] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:41 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:42.207560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.225953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa08ad-d3b2-4e0e-813d-0ca9acc510f7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.233891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.235048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.235526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.236004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.237421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.240524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.260788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 988/1963] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:42.266034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.285279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc803ec-e799-4c91-b350-498dff1aeda5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.294362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.295505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.298951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.298951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.298951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.301132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.323594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 989/1964] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:42.330341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.348732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458cbca-2153-4c13-8d57-2f3468ed6ec5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.352790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5ad8809e144bfea78468e44f1f79e5', 'request_id': 'req-e135f671-d03f-4949-a1a4-c0eccaf1b16f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:42.360960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5ad8809e144bfea78468e44f1f79e5', 'request_id': 'req-e135f671-d03f-4949-a1a4-c0eccaf1b16f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:42.450032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.451325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.451734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.452156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] PATH_INFO: `/v3/projects/0ab2e5a8f8c1436289b519d108694734/users/8c23233891e84931867c27b5ebebfb46/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.472513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab2e5a8f8c1436289b519d108694734, user_id=8c23233891e84931867c27b5ebebfb46, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.473651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5ad8809e144bfea78468e44f1f79e5', 'request_id': 'req-e135f671-d03f-4949-a1a4-c0eccaf1b16f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:42.475302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.488365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] The token cache is being invalidate because project 8f5ad8809e144bfea78468e44f1f79e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:42.488796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:42.489640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e135f671-d03f-4949-a1a4-c0eccaf1b16f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:42.491718 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 976/1965] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:42 2022] DELETE /identity/v3/projects/8f5ad8809e144bfea78468e44f1f79e5 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:42.497308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a70d9e-f662-4a5b-8c36-fd57e0fd5f7d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.497724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a70d9e-f662-4a5b-8c36-fd57e0fd5f7d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.498100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a70d9e-f662-4a5b-8c36-fd57e0fd5f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.518479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d3a70d9e-f662-4a5b-8c36-fd57e0fd5f7d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:42.520201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d3a70d9e-f662-4a5b-8c36-fd57e0fd5f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:42.530051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 977/1966] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:42.538752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.539048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.539334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.559881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:42.561202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:42.569698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 978/1967] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:42.575776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.586482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ffb0f-1602-4cdd-be99-5ea7afb0bedb None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.591905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.592696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.592987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.593264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.594032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:42.594346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.596025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8d9d1-62ab-4445-8eb9-912dc2e6b21b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.610215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 990/1968] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:42 2022] PUT /identity/v3/projects/0ab2e5a8f8c1436289b519d108694734/users/8c23233891e84931867c27b5ebebfb46/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:42.624325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.624623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 979/1969] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:42.640626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.640906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.641261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.675893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] MFA Rules not processed for user `8c23233891e84931867c27b5ebebfb46`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:42.677358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:42.779056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 980/1970] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:42.795936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:42.798856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:42.828673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664fb1-2d0d-4408-8053-d38003a222ab admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.838815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.839864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.840391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.841086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.842251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.845344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.930483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e156c-255b-4b15-ab70-1f9831197db7 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:42.935025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:42.936404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:42.936689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:42.937130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:42.948327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:42.949889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:42.962101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 981/1971] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:43.027409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 991/1972] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:42 2022] POST /identity/v3/projects => generated 379 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:43.033050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:43.039821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:43.047233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f222a41-8fcc-4f80-8b31-bf33812184a4 service glance] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:43.153731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:43.160521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:43.160521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:43.160521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:43.160521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:43.160521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:43.333806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 982/1973] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:43 2022] POST /identity/v3/users => generated 290 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:43.339467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:43.352574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e86a-62d3-4500-a465-1c0e540f2b10 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:43.359811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:43.360813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:43.361220 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:43.361705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:43.363128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:43.366725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:43.385429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 992/1974] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:43.391047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:43.408501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaf5dc-3c6f-4a0b-a71d-1b2eda77cb0c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:43.417106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d58d5f-1c73-4563-ad08-2b939aad481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:43.418248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d58d5f-1c73-4563-ad08-2b939aad481d admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:43.418660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d58d5f-1c73-4563-ad08-2b939aad481d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:43.419159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d58d5f-1c73-4563-ad08-2b939aad481d admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:43.420527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d58d5f-1c73-4563-ad08-2b939aad481d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:43.423264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d58d5f-1c73-4563-ad08-2b939aad481d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:43.442098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 983/1975] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:43.446616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:43.457569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bc9f6-5813-45b7-ad13-0c71d6f81173 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:43.462547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:43.463527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:43.463807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:43.464081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] PATH_INFO: `/v3/projects/1e643a38dc72420f9201da4240c290b9/users/c894c44611f6469a905fad19ea43229e/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:43.481777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e643a38dc72420f9201da4240c290b9, user_id=c894c44611f6469a905fad19ea43229e, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:43.484613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:43.621746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 993/1976] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:43 2022] PUT /identity/v3/projects/1e643a38dc72420f9201da4240c290b9/users/c894c44611f6469a905fad19ea43229e/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:43.632691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:43.633071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:43.633428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:43.664547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] MFA Rules not processed for user `c894c44611f6469a905fad19ea43229e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:43.665831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:43.761104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 984/1977] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:43.766133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:43.851443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad518fbf-2c41-4ee3-b702-c0cd4a40a902 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:43.859775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:43.860914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:43.861374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:43.861759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:43.862674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:43.867305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:43.970121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 994/1978] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:43 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:43.977070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:43.995913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac492bc-ae3b-449b-bbf1-3024ef76a8ef None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:44.119134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:44.120347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.121304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.121737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.122820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:44.125404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:44.240312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 985/1979] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:43 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:44.247188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:44.264931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2984-3cd7-47ea-b854-97d242e64431 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:44.274863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:44.276056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.276604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.277107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.278521 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:44.281799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:44.302928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 995/1980] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:27:44.309903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:44.321229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf107d77-3a6b-4a6e-a4e5-3b98182d51c5 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:44.326320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:44.327259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.327545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.327825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] PATH_INFO: `/v3/projects/7c66711bd72242d9a6d41b748835beb2/users/39d5eae659df4741aad05ca7a425dcf3/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.345931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c66711bd72242d9a6d41b748835beb2, user_id=39d5eae659df4741aad05ca7a425dcf3, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:44.348924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:44.472352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 986/1981] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:44 2022] PUT /identity/v3/projects/7c66711bd72242d9a6d41b748835beb2/users/39d5eae659df4741aad05ca7a425dcf3/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:44.476923 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:44.488665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128425fa-2589-4414-b97e-075d9a80d842 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:44.553815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba009de1-aae0-4bd4-82fc-eec8f2e72575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:44.554531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009de1-aae0-4bd4-82fc-eec8f2e72575 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.554819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009de1-aae0-4bd4-82fc-eec8f2e72575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.555103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009de1-aae0-4bd4-82fc-eec8f2e72575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.555859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba009de1-aae0-4bd4-82fc-eec8f2e72575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:44.557704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba009de1-aae0-4bd4-82fc-eec8f2e72575 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:44.569178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 996/1982] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:44.574410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:44.592829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bed554-c66e-4005-9f7a-c2887b4889f3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:44.601122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:44.602604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.604076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.604076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] PATH_INFO: `/v3/projects/7c66711bd72242d9a6d41b748835beb2/users/39d5eae659df4741aad05ca7a425dcf3/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.607708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c66711bd72242d9a6d41b748835beb2, user_id=39d5eae659df4741aad05ca7a425dcf3, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:44.613303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:44.627054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:44.632649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 987/1983] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:44 2022] PUT /identity/v3/projects/7c66711bd72242d9a6d41b748835beb2/users/39d5eae659df4741aad05ca7a425dcf3/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:44.645650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.645944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.646225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.661501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:44.677999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] MFA Rules not processed for user `39d5eae659df4741aad05ca7a425dcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:44.679277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:44.761564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 997/1984] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:44.774035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:44.776176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:44.844242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d76d7-66db-46f9-bc14-d4518a13c97f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:44.848334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7528b-e0be-43b4-94c3-0064c838738e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:44.849018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7528b-e0be-43b4-94c3-0064c838738e service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:44.849303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7528b-e0be-43b4-94c3-0064c838738e service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:44.849581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7528b-e0be-43b4-94c3-0064c838738e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:44.862975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7528b-e0be-43b4-94c3-0064c838738e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:44.866042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7528b-e0be-43b4-94c3-0064c838738e service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:44.885912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 988/1985] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:45.484709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:45.632561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:45.749986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ffe58-ac9d-422c-ae1b-e1790dde06db None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:45.755104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:45.756054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:45.756897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:45.757176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] PATH_INFO: `/v3/users/c894c44611f6469a905fad19ea43229e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:45.758688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c894c44611f6469a905fad19ea43229e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:45.760799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:45.998866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:46.066671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c894c44611f6469a905fad19ea43229e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:46.077256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c894c44611f6469a905fad19ea43229e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:46.078453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c894c44611f6469a905fad19ea43229e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:46.095476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c894c44611f6469a905fad19ea43229e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:46.096265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c894c44611f6469a905fad19ea43229e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:46.233830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 998/1986] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:45 2022] DELETE /identity/v3/users/c894c44611f6469a905fad19ea43229e => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:46.241755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:46.242263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:46.242807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:46.274276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:46.276730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:46.379056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 989/1987] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:46.394215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:46.396436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:46.485310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712012f-47fa-4655-bbe1-478665331de3 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:46.489528 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:46.490305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:46.490645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:46.490957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:46.503810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:46.505671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:46.520275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 999/1988] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:47.124162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:47.167410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:47.170278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:47.195008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5df5fe-fefc-48fc-98d4-8508c686380d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:47.207674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:47.209909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:47.209909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:47.209909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] PATH_INFO: `/v3/projects/1e643a38dc72420f9201da4240c290b9` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:47.211413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e643a38dc72420f9201da4240c290b9)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:47.213854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:47.405705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e643a38dc72420f9201da4240c290b9', 'request_id': 'req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:47.418604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e643a38dc72420f9201da4240c290b9', 'request_id': 'req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:47.560244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e643a38dc72420f9201da4240c290b9', 'request_id': 'req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:47.583799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] The token cache is being invalidate because project 1e643a38dc72420f9201da4240c290b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:47.584365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:47.585668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:47.587790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 990/1989] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:47 2022] DELETE /identity/v3/projects/1e643a38dc72420f9201da4240c290b9 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:47.592915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:47.611386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ca06b-5d8b-4f43-93c4-19e23f43a255 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:47.755959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:47.758226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:47.758754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:47.759231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] PATH_INFO: `/v3/users/39d5eae659df4741aad05ca7a425dcf3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:47.761567 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d5eae659df4741aad05ca7a425dcf3)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:47.765539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:47.894179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:48.119861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d5eae659df4741aad05ca7a425dcf3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:48.131567 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d5eae659df4741aad05ca7a425dcf3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:48.132450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d5eae659df4741aad05ca7a425dcf3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:48.149801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d5eae659df4741aad05ca7a425dcf3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:48.150634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8f4a754c-952c-4441-8bd0-964fd7e861e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d5eae659df4741aad05ca7a425dcf3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:48.892450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1000/1990] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:47 2022] DELETE /identity/v3/users/39d5eae659df4741aad05ca7a425dcf3 => generated 0 bytes in 1299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:49.666578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:49.668818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:49.687599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a2f2c-55a6-41ed-9e2d-f865a92deb6d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:49.777430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:49.778909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:49.779417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:49.779924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] PATH_INFO: `/v3/projects/7c66711bd72242d9a6d41b748835beb2` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:49.782312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c66711bd72242d9a6d41b748835beb2)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:49.786336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:49.981395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c66711bd72242d9a6d41b748835beb2', 'request_id': 'req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:49.991269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c66711bd72242d9a6d41b748835beb2', 'request_id': 'req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:50.089389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c66711bd72242d9a6d41b748835beb2', 'request_id': 'req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:50.102889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] The token cache is being invalidate because project 7c66711bd72242d9a6d41b748835beb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:50.103280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:50.104345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-612a9d43-bb65-4fda-a6d6-0ab00fa2bb5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:50.106509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 991/1991] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:49 2022] DELETE /identity/v3/projects/7c66711bd72242d9a6d41b748835beb2 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:50.115946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.120872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.120872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.148695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:50.150943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:50.282611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1001/1992] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:50.285758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.286261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.287211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.333318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:50.336156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:50.360560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 992/1993] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:50.363365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:50.378974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df22ab-b8c9-44b7-8564-355a5091cc12 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:50.397397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:50.397397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.397701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.397922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.398941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:50.399415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:50.401466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:50.424406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1002/1994] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:27:50.426210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:50.438151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd40d9f-3a17-4aa4-91f7-3676a35e7292 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:50.445094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:50.448398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.448398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.448398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.448398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:50.450589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:50.493118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:50.495924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:50.578458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd47aa-c885-4c82-9ead-e906ceb61639 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:50.583552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:50.584561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.585010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.585439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.588148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:50.599240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 993/1995] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:50 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:50.605573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:50.622993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:50.702126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:50.705424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:50.720627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1003/1996] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:27:50.747245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34728879-4ffe-4dbb-84fa-ba6b6eae0252 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:50.755630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:50.757316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.768512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.768512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.768512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:50.768512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:50.923853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 994/1997] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:50 2022] POST /identity/v3/users => generated 287 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:50.929546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:50.949430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b52123-156f-49c8-bcda-58b5e1c0ddc3 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:50.957448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:50.963781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:50.963781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:50.963781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:50.963781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:50.965072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:50.985703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1004/1998] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:50.990196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:51.001616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce958abc-16ea-4ed4-a648-d31ab4bdc1d9 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:51.007502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2d22a-c81a-47b7-a7da-3d83e55da73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:51.008224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2d22a-c81a-47b7-a7da-3d83e55da73c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.008593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2d22a-c81a-47b7-a7da-3d83e55da73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.008883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2d22a-c81a-47b7-a7da-3d83e55da73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.009693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2d22a-c81a-47b7-a7da-3d83e55da73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:51.011361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2d22a-c81a-47b7-a7da-3d83e55da73c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:51.025240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 995/1999] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:51.030718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:51.049540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c969-7a32-43c0-8aa1-f4bb92edcec8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:51.096606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:51.096606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.096606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.096606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] PATH_INFO: `/v3/projects/6579616ed5b449588eb49e4e3c3b69d8/users/f056cd29e92a4d54818f9888f57e57b0/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.097482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6579616ed5b449588eb49e4e3c3b69d8, user_id=f056cd29e92a4d54818f9888f57e57b0, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:51.097482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:51.200600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1005/2000] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:51 2022] PUT /identity/v3/projects/6579616ed5b449588eb49e4e3c3b69d8/users/f056cd29e92a4d54818f9888f57e57b0/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:51.215945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.216312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.216652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.250893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] MFA Rules not processed for user `f056cd29e92a4d54818f9888f57e57b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:51.252049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:51.334858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 996/2001] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:51.341248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:51.426981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899601e-40ed-4761-aded-80b500e4be49 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:51.432955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:51.433640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.433924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.434213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.434936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:51.437078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:51.577786 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1006/2002] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:51 2022] POST /identity/v3/projects => generated 373 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:51.584997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:51.604772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767064e-6b99-4c29-aea4-b03a84468836 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:51.677097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:51.677787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.678141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.678468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.679266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:51.681713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:51.814942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 997/2003] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:51 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:51.816203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:51.828840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb14d8d-cf4a-4375-98fb-878a9b7f95fc admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:51.835499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:51.837060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.839028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.839028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.839028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:51.840785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:51.854253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1007/2004] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:51.859485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:51.879270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0659-7cda-458a-8e88-55e08f81f022 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:51.890536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:51.891558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:51.892087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:51.892634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] PATH_INFO: `/v3/projects/949e6d14f4b1429c92a6a28e91af0c37/users/f3e5d63c93f44e7ca5e3355a9fd303f8/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:51.915840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949e6d14f4b1429c92a6a28e91af0c37, user_id=f3e5d63c93f44e7ca5e3355a9fd303f8, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:51.918781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:52.070533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 998/2005] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:51 2022] PUT /identity/v3/projects/949e6d14f4b1429c92a6a28e91af0c37/users/f3e5d63c93f44e7ca5e3355a9fd303f8/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:52.081825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:52.093317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863efcf1-6279-4b49-937e-0ae8d4489cf1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:52.107834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:52.183531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb078e2-454e-4d98-873b-596ea3c6f68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:52.184505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb078e2-454e-4d98-873b-596ea3c6f68c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:52.185307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb078e2-454e-4d98-873b-596ea3c6f68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:52.185808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb078e2-454e-4d98-873b-596ea3c6f68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:52.187090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb078e2-454e-4d98-873b-596ea3c6f68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:52.190726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb078e2-454e-4d98-873b-596ea3c6f68c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:52.204319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1008/2006] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:52.211239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:52.222523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a31e6f-073e-4f41-8f55-aaa1f1095e18 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:52.229027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:52.229984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:52.230235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:52.230511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] PATH_INFO: `/v3/projects/949e6d14f4b1429c92a6a28e91af0c37/users/f3e5d63c93f44e7ca5e3355a9fd303f8/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:52.232704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=949e6d14f4b1429c92a6a28e91af0c37, user_id=f3e5d63c93f44e7ca5e3355a9fd303f8, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:52.235435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:52.247934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 999/2007] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:52 2022] PUT /identity/v3/projects/949e6d14f4b1429c92a6a28e91af0c37/users/f3e5d63c93f44e7ca5e3355a9fd303f8/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:52.267715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:52.268061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:52.268383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:52.301453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] MFA Rules not processed for user `f3e5d63c93f44e7ca5e3355a9fd303f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:52.303428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:52.387998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1009/2008] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:52.401460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:52.403350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:52.478350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d139bb-651d-4d13-8ed7-d87cefdada1b admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:52.482597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488cbd2-d525-4dad-97ff-15b5a5ecd6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:52.483283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488cbd2-d525-4dad-97ff-15b5a5ecd6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:52.483569 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488cbd2-d525-4dad-97ff-15b5a5ecd6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:52.483852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488cbd2-d525-4dad-97ff-15b5a5ecd6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:52.550160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488cbd2-d525-4dad-97ff-15b5a5ecd6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:52.552241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488cbd2-d525-4dad-97ff-15b5a5ecd6e9 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:52.565953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1000/2009] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:27:53.715966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:53.721092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:53.811817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40d453-1186-4338-93d3-c21b816526bd None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:53.817283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:53.818967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:53.819299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:53.819673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] PATH_INFO: `/v3/users/59823bde2ed74526a78ea40b737b88e7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:53.821271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59823bde2ed74526a78ea40b737b88e7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:53.823369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:54.133432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59823bde2ed74526a78ea40b737b88e7'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:54.141203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59823bde2ed74526a78ea40b737b88e7'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:54.142100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59823bde2ed74526a78ea40b737b88e7'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:54.157336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59823bde2ed74526a78ea40b737b88e7'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:54.158086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59823bde2ed74526a78ea40b737b88e7'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:54.284848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1010/2010] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:53 2022] DELETE /identity/v3/users/59823bde2ed74526a78ea40b737b88e7 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:54.288769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:54.291678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:54.292375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:54.292375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:54.321190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:54.321190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:54.419099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1001/2011] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:54.433453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:54.436782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:54.530923 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1706b24-f889-41aa-b177-41906a2fc897 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:54.538289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:54.539464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:54.539981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:54.540541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:54.552135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:54.553658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:54.565107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1011/2012] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:55.287568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:55.289726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:55.316331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9741c38-74a6-46d4-8d43-dd133e6ad397 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:55.326497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:55.327981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:55.328495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:55.328948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] PATH_INFO: `/v3/projects/cabeb8e0571c4dd0b3ffdd0bbf4fb9f4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:55.331009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabeb8e0571c4dd0b3ffdd0bbf4fb9f4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:55.334004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:55.474846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabeb8e0571c4dd0b3ffdd0bbf4fb9f4', 'request_id': 'req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:55.482257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabeb8e0571c4dd0b3ffdd0bbf4fb9f4', 'request_id': 'req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:55.583458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabeb8e0571c4dd0b3ffdd0bbf4fb9f4', 'request_id': 'req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:55.608616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] The token cache is being invalidate because project cabeb8e0571c4dd0b3ffdd0bbf4fb9f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:55.609049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:55.610364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:55.612884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1002/2013] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:55 2022] DELETE /identity/v3/projects/cabeb8e0571c4dd0b3ffdd0bbf4fb9f4 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:55.617990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:55.635377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7773-589b-4320-b6ee-5775425fda1b service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:55.736228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:55.737582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:55.738083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:55.738438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] PATH_INFO: `/v3/users/818876a8d1a942dc8b71ea13de8cb474` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:55.740027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818876a8d1a942dc8b71ea13de8cb474)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:55.742798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:55.837896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:56.030262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818876a8d1a942dc8b71ea13de8cb474'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.039375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818876a8d1a942dc8b71ea13de8cb474'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.040269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818876a8d1a942dc8b71ea13de8cb474'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.055044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818876a8d1a942dc8b71ea13de8cb474'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.055837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818876a8d1a942dc8b71ea13de8cb474'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.170073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1012/2014] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:55 2022] DELETE /identity/v3/users/818876a8d1a942dc8b71ea13de8cb474 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:56.319685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:56.322405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:56.346903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d700f1-8b97-4d83-9c50-5e6ddba1e59c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:56.467312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:56.468929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:56.469539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:56.470091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] PATH_INFO: `/v3/users/f056cd29e92a4d54818f9888f57e57b0` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:56.472240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f056cd29e92a4d54818f9888f57e57b0)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:56.476186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:56.761703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:56.763222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:56.767478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f056cd29e92a4d54818f9888f57e57b0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.768895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f056cd29e92a4d54818f9888f57e57b0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.779010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f056cd29e92a4d54818f9888f57e57b0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.780915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db2e09-a9fe-4ee2-b466-701f08da32c9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:56.788128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:56.793727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:56.793727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:56.793727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] PATH_INFO: `/v3/users/afb5a341bf5e460abfc8df409c2a1d61` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:56.793727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb5a341bf5e460abfc8df409c2a1d61)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:56.795694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:56.798631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f056cd29e92a4d54818f9888f57e57b0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.800127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f056cd29e92a4d54818f9888f57e57b0'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:56.899815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1003/2015] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:56 2022] DELETE /identity/v3/users/f056cd29e92a4d54818f9888f57e57b0 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:56.903971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:56.905535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.021596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3367-fab2-4105-bddc-bc24bbb50cc0 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:57.026749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:57.027656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.027946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.028225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] PATH_INFO: `/v3/projects/dd39021a4ea448aaa1c5dd787a3ea9eb` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.029650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd39021a4ea448aaa1c5dd787a3ea9eb)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:57.031694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:57.052795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb5a341bf5e460abfc8df409c2a1d61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.061383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb5a341bf5e460abfc8df409c2a1d61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.062152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb5a341bf5e460abfc8df409c2a1d61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.080999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb5a341bf5e460abfc8df409c2a1d61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.080999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2913d389-d964-481f-934f-f995b3627d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb5a341bf5e460abfc8df409c2a1d61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.248665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd39021a4ea448aaa1c5dd787a3ea9eb', 'request_id': 'req-047927c5-d651-4ad6-aeb2-5402564126fd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.251945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1013/2016] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:56 2022] DELETE /identity/v3/users/afb5a341bf5e460abfc8df409c2a1d61 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:27:57.254182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee6215-11c1-4f08-b096-eabbde7b14f2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.254684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee6215-11c1-4f08-b096-eabbde7b14f2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.255157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee6215-11c1-4f08-b096-eabbde7b14f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.265671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd39021a4ea448aaa1c5dd787a3ea9eb', 'request_id': 'req-047927c5-d651-4ad6-aeb2-5402564126fd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.279172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-dcee6215-11c1-4f08-b096-eabbde7b14f2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:57.281373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dcee6215-11c1-4f08-b096-eabbde7b14f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.363507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1014/2017] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:57.365083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.368815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.373197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.389276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:57.390549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.399835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1015/2018] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:57.401990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:57.404233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.454071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd39021a4ea448aaa1c5dd787a3ea9eb', 'request_id': 'req-047927c5-d651-4ad6-aeb2-5402564126fd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.468817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] The token cache is being invalidate because project dd39021a4ea448aaa1c5dd787a3ea9eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:57.469658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.470649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:57.472323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:57.473286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1004/2019] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:56 2022] DELETE /identity/v3/projects/dd39021a4ea448aaa1c5dd787a3ea9eb => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:57.475480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:57.477239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624e887-58da-4d17-b8fa-55130aa2728f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:57.482119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.482324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:57.482324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.482324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.482760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.484413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.499408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:57.501020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f8af5-4986-4b9e-be49-f9bdfa9fb847 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:57.512417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1016/2020] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:57.514697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa34a-1fa3-4ab9-9421-e42a87da2771 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.514697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa34a-1fa3-4ab9-9421-e42a87da2771 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.514697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa34a-1fa3-4ab9-9421-e42a87da2771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.547709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7acaa34a-1fa3-4ab9-9421-e42a87da2771 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:57.549114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7acaa34a-1fa3-4ab9-9421-e42a87da2771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.564009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1017/2021] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:57.572317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047927c5-d651-4ad6-aeb2-5402564126fd admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:57.578768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.579241 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.579713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.585007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:57.586255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.586736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.589936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.590422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.612233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:57.612543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:57.614489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:57.615267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:57.626838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1018/2022] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:27:57.633081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:57.635615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1005/2023] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:27:57.654940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b53a9f-dace-4a20-90da-d278f641b071 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:57.664171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:57.666682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.666682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.672424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:57.672704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.677278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:27:57.677278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:57.677278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:57.691298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1019/2024] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:27:57.698507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:57.715429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf9923-b856-48d2-978d-6fd6be4c5831 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:57.725373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:57.727726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:57.727726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:57.727928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:57.729486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:57.732889 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:57.863101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1006/2025] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:57 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:27:57.865892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:57.886914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0289cda-1bb6-4b8b-b5c1-ed54b2f866c4 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.001559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.002315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.002601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.002880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.003612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.006927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.126362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1020/2026] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:57 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:58.132334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.143649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9d12c-4148-4a96-add5-76d56ba25b0e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.149908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.150607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.150894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.151799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.152927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.154608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.180337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1007/2027] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:58.187458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.200464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5967a6-e5c5-4313-85e5-0a9f2527c772 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.206293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.207166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.219645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.219645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.219645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.219645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.235286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1021/2028] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:58.241999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.259012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a82624-9d07-49bc-9ea6-57d881fa139d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.264125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.267372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.267372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.267372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] PATH_INFO: `/v3/projects/24a5fd4d3bb44d5d9d89e362c6630882/users/14681df64b9c4a6c818ecf6775508122/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.320519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a5fd4d3bb44d5d9d89e362c6630882, user_id=14681df64b9c4a6c818ecf6775508122, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.321441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.324919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:58.330965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.355196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6b310-4855-44aa-8489-fe770bdf4bf0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.364732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.366283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.366575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.367014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] PATH_INFO: `/v3/projects/6579616ed5b449588eb49e4e3c3b69d8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.368941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6579616ed5b449588eb49e4e3c3b69d8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.371415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.495484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1008/2029] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:58 2022] PUT /identity/v3/projects/24a5fd4d3bb44d5d9d89e362c6630882/users/14681df64b9c4a6c818ecf6775508122/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:58.496378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.498390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:58.571923 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6579616ed5b449588eb49e4e3c3b69d8', 'request_id': 'req-12a44574-8cb5-4e09-a840-e1fa8eb52479'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.574472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec133be-1edb-42db-8482-7a7b363c781e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.580907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6579616ed5b449588eb49e4e3c3b69d8', 'request_id': 'req-12a44574-8cb5-4e09-a840-e1fa8eb52479'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.643907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.644881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.645189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.645479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] PATH_INFO: `/v3/projects/144eca287dc94b45a2dd6e40b8772c7b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.646885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=144eca287dc94b45a2dd6e40b8772c7b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.648891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.687199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6579616ed5b449588eb49e4e3c3b69d8', 'request_id': 'req-12a44574-8cb5-4e09-a840-e1fa8eb52479'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.701715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] The token cache is being invalidate because project 6579616ed5b449588eb49e4e3c3b69d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:58.702011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.702725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-12a44574-8cb5-4e09-a840-e1fa8eb52479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.704446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1022/2030] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:58 2022] DELETE /identity/v3/projects/6579616ed5b449588eb49e4e3c3b69d8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:58.705803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.706082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.706350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.744908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] MFA Rules not processed for user `14681df64b9c4a6c818ecf6775508122`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:27:58.746296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:58.820445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144eca287dc94b45a2dd6e40b8772c7b', 'request_id': 'req-dbc9d79a-c87b-48dc-b674-af7f2812e375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.830915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144eca287dc94b45a2dd6e40b8772c7b', 'request_id': 'req-dbc9d79a-c87b-48dc-b674-af7f2812e375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.834444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1023/2031] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:27:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:27:58.836816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.842050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:58.853290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37963302-e9c0-4ade-a2f7-8dd2fd793167 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.922269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144eca287dc94b45a2dd6e40b8772c7b', 'request_id': 'req-dbc9d79a-c87b-48dc-b674-af7f2812e375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.938481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.939452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.939751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.940061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] PATH_INFO: `/v3/users/f3e5d63c93f44e7ca5e3355a9fd303f8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.940760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] The token cache is being invalidate because project 144eca287dc94b45a2dd6e40b8772c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:27:58.941226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.942284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e5d63c93f44e7ca5e3355a9fd303f8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.942406 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:58.944834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1009/2032] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:27:58 2022] DELETE /identity/v3/projects/144eca287dc94b45a2dd6e40b8772c7b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:58.946002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:58.946121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:58.948309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:58.969205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9d79a-c87b-48dc-b674-af7f2812e375 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:58.977052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:58.978119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:58.978585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:58.979035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:58.980219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:58.983767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:59.113158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1010/2033] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:27:58 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:59.117639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:59.119697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:59.196904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272bf2f-3383-4e93-a620-2961d1231edc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:59.202087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:59.204078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:59.204078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:59.204078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] PATH_INFO: `/v3/users/112a8b82cd504dde829c0c880c268a1d` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:59.205633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=112a8b82cd504dde829c0c880c268a1d)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:59.207711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:59.211636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e5d63c93f44e7ca5e3355a9fd303f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.221099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e5d63c93f44e7ca5e3355a9fd303f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.223049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e5d63c93f44e7ca5e3355a9fd303f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.242353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e5d63c93f44e7ca5e3355a9fd303f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.243406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e5d63c93f44e7ca5e3355a9fd303f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.421679 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1024/2034] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:58 2022] DELETE /identity/v3/users/f3e5d63c93f44e7ca5e3355a9fd303f8 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:59.423455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:59.427601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:59.524396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284a082-636d-4083-bac8-153348c1b11c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:59.535267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:59.536393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:59.536795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:59.537787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:59.538028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:59.541084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:59.614755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112a8b82cd504dde829c0c880c268a1d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.615877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112a8b82cd504dde829c0c880c268a1d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.629017 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112a8b82cd504dde829c0c880c268a1d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.649230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112a8b82cd504dde829c0c880c268a1d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.650012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112a8b82cd504dde829c0c880c268a1d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:27:59.697189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1025/2035] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:27:59 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:27:59.705302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:59.709111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:27:59.733336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994a8e-61e2-4dc7-94d8-58e5063c2d36 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:59.745546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1011/2036] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:27:59 2022] DELETE /identity/v3/users/112a8b82cd504dde829c0c880c268a1d => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:27:59.750972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:59.752402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:59.752912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:59.753371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:59.755589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:59.759079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:27:59.784000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1026/2037] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:27:59.789468 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:27:59.791826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:59.941469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827f9e-d512-4266-ac91-5190d7b264b3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:27:59.948162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:27:59.949155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:27:59.949437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:27:59.949733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] PATH_INFO: `/v3/projects/730234dfbd1347a3abc4f77fb9ca2f29/users/0d40e7639dcf4965969f96b15ac25527/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:27:59.969143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:27:59.970887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=730234dfbd1347a3abc4f77fb9ca2f29, user_id=0d40e7639dcf4965969f96b15ac25527, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:27:59.975502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:00.089580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1012/2038] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:27:59 2022] PUT /identity/v3/projects/730234dfbd1347a3abc4f77fb9ca2f29/users/0d40e7639dcf4965969f96b15ac25527/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:00.094200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:00.107655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8a7b7-94aa-46e6-a29e-1079a81ab026 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:00.174194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:00.174899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.175185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.175471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.176263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:00.178514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:00.190076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1027/2039] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:00.196939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:00.220603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:00.231351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:00.241657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c07a-d230-4414-9425-39f03e964362 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:00.255495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1306ad4-46c4-447c-bec0-1514a4b0a202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:00.257238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f7e0-4d2e-4d40-96cc-038413e77efc admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:00.258640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1306ad4-46c4-447c-bec0-1514a4b0a202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.259257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1306ad4-46c4-447c-bec0-1514a4b0a202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.259748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1306ad4-46c4-447c-bec0-1514a4b0a202 admin admin] PATH_INFO: `/v3/projects/730234dfbd1347a3abc4f77fb9ca2f29/users/0d40e7639dcf4965969f96b15ac25527/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.263035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:00.264048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1306ad4-46c4-447c-bec0-1514a4b0a202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=730234dfbd1347a3abc4f77fb9ca2f29, user_id=0d40e7639dcf4965969f96b15ac25527, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:00.270372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.270372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.270372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] PATH_INFO: `/v3/projects/949e6d14f4b1429c92a6a28e91af0c37` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.270372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=949e6d14f4b1429c92a6a28e91af0c37)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:00.270372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:00.304939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1306ad4-46c4-447c-bec0-1514a4b0a202 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:00.325359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1013/2040] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:00 2022] PUT /identity/v3/projects/730234dfbd1347a3abc4f77fb9ca2f29/users/0d40e7639dcf4965969f96b15ac25527/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:00.346207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.346720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.347226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.399859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] MFA Rules not processed for user `0d40e7639dcf4965969f96b15ac25527`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:00.402059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:00.438700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949e6d14f4b1429c92a6a28e91af0c37', 'request_id': 'req-f0012c7c-83f1-488a-978c-3f05dc94a082'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.447046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949e6d14f4b1429c92a6a28e91af0c37', 'request_id': 'req-f0012c7c-83f1-488a-978c-3f05dc94a082'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.494664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1014/2041] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:00.508625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:00.510210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:00.557191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949e6d14f4b1429c92a6a28e91af0c37', 'request_id': 'req-f0012c7c-83f1-488a-978c-3f05dc94a082'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.583748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] The token cache is being invalidate because project 949e6d14f4b1429c92a6a28e91af0c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:00.584494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.585471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.589987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1028/2042] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:00 2022] DELETE /identity/v3/projects/949e6d14f4b1429c92a6a28e91af0c37 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:00.595910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:00.598614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:00.632573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3c335-96be-426e-a2ff-7eb8e50e83c8 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:00.678348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0012c7c-83f1-488a-978c-3f05dc94a082 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:00.683978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:00.684976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.685293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.685630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] PATH_INFO: `/v3/projects/77cd77b505b54ee4873b1a3b5caac059` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.687058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=77cd77b505b54ee4873b1a3b5caac059)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:00.689187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:00.731617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:00.732503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.732965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.733470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.735919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:00.846073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:00.848324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a3093-c458-4371-8bdd-baa95b8a8729 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:00.854407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77cd77b505b54ee4873b1a3b5caac059', 'request_id': 'req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.866471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1015/2043] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:28:00.869574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf90f-1722-4b37-ab1d-13eb16298bfc None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:00.870537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf90f-1722-4b37-ab1d-13eb16298bfc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:00.871141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf90f-1722-4b37-ab1d-13eb16298bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:00.873153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77cd77b505b54ee4873b1a3b5caac059', 'request_id': 'req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:00.906990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-59ccf90f-1722-4b37-ab1d-13eb16298bfc None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:00.909273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-59ccf90f-1722-4b37-ab1d-13eb16298bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:00.996809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77cd77b505b54ee4873b1a3b5caac059', 'request_id': 'req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.005242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1016/2044] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:01.024990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.024990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.024990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.025478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] The token cache is being invalidate because project 77cd77b505b54ee4873b1a3b5caac059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:01.025478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.026548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.029705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1029/2045] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:00 2022] DELETE /identity/v3/projects/77cd77b505b54ee4873b1a3b5caac059 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:01.037440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.040166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:01.060023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:01.062618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:01.064849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247dda-a5e6-4a73-8c70-58a7a2670c1f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.078560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1017/2046] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:01.083460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.085136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.085366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.085691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.086074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] PATH_INFO: `/v3/users/7ccd80acfcbe402889b79e903887388a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.090603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ccd80acfcbe402889b79e903887388a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.104013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:01.104983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b8471-91dc-4ca6-86b2-846f2551ad21 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.117826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.119249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.119857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.120225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.121665 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:01.122219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.125833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:01.144693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:01.154854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1018/2047] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:01.160205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.171303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf63da-cdde-4e3a-bb14-e5967ca10066 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.178188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.178875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.179158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.179431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.180128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.182512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:01.422057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1019/2048] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:01 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:01.427984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.437640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ccd80acfcbe402889b79e903887388a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.450448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ccd80acfcbe402889b79e903887388a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.451449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ccd80acfcbe402889b79e903887388a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.471038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ccd80acfcbe402889b79e903887388a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.471886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ccd80acfcbe402889b79e903887388a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:01.526027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52301e1e-0994-4000-a2f4-03c1c61e788f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.531577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.532601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.533015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.533378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.534257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.537205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:01.562376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1030/2049] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:01 2022] DELETE /identity/v3/users/7ccd80acfcbe402889b79e903887388a => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:01.655628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1020/2050] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:01 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:01.662098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.664477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:01.687510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:01.755982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4882a-67a1-4dff-aa85-2c4b72c60c59 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.762326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.763302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.763627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.763909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.764688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.766315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:01.782169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1031/2051] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:01.783480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.797784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35bd9-1f4b-4270-bd7c-003935521ed6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.806325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edc6d0-837a-4bda-9d36-99051cc7591b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.807523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc6d0-837a-4bda-9d36-99051cc7591b admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.808034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc6d0-837a-4bda-9d36-99051cc7591b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.808602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc6d0-837a-4bda-9d36-99051cc7591b admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.810011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edc6d0-837a-4bda-9d36-99051cc7591b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.813660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edc6d0-837a-4bda-9d36-99051cc7591b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:01.847013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1021/2052] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:01.851549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:01.873234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af81e8a-f27c-4757-93ce-4a3c8abdeb65 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:01.880556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:01.881605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:01.882129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:01.882299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] PATH_INFO: `/v3/projects/8d2b00583cd54625bda4fa80d98ff191/users/f1e9ad843d364ae0bac05703288d7041/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:01.915957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2b00583cd54625bda4fa80d98ff191, user_id=f1e9ad843d364ae0bac05703288d7041, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:01.921084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:02.088152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1032/2053] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:01 2022] PUT /identity/v3/projects/8d2b00583cd54625bda4fa80d98ff191/users/f1e9ad843d364ae0bac05703288d7041/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:02.106960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:02.107174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:02.107485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:02.137235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] MFA Rules not processed for user `f1e9ad843d364ae0bac05703288d7041`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:02.138576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:02.216247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1022/2054] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:02.230901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:02.233865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:02.274512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:02.276158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:02.351951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df0b6-818e-4601-b5c6-06d8fc0a7195 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:02.356714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:02.357481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:02.357695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:02.358092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:02.372550 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:02.374534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:02.389557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1033/2055] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:02.402868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8556a6c-0637-45fb-9c8f-34989b58712f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:02.408585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:02.409754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:02.410307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:02.410589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] PATH_INFO: `/v3/projects/484042f0149d4518bb5b4005626ccc24` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:02.417546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=484042f0149d4518bb5b4005626ccc24)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:02.419551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:02.421337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:02.440389 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bbd2c-6d0c-4975-9d5b-d68e706597b2 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:02.450221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:02.451958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:02.452595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:02.452951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] PATH_INFO: `/v3/users/f1e9ad843d364ae0bac05703288d7041` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:02.455287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e9ad843d364ae0bac05703288d7041)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:02.459296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:02.614914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484042f0149d4518bb5b4005626ccc24', 'request_id': 'req-19dce60e-f361-48d7-b32c-b3e99fd26d6d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.623510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484042f0149d4518bb5b4005626ccc24', 'request_id': 'req-19dce60e-f361-48d7-b32c-b3e99fd26d6d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.739119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484042f0149d4518bb5b4005626ccc24', 'request_id': 'req-19dce60e-f361-48d7-b32c-b3e99fd26d6d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.756144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e9ad843d364ae0bac05703288d7041'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.760054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] The token cache is being invalidate because project 484042f0149d4518bb5b4005626ccc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:02.760634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.761921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.764172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e9ad843d364ae0bac05703288d7041'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.770845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1023/2056] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:02 2022] DELETE /identity/v3/projects/484042f0149d4518bb5b4005626ccc24 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:02.771056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e9ad843d364ae0bac05703288d7041'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.771225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:02.771225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:02.779736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e9ad843d364ae0bac05703288d7041'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.780507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-64b3a30e-35a4-4f2b-8775-8c96c90da047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e9ad843d364ae0bac05703288d7041'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:02.802423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:02.859363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce60e-f361-48d7-b32c-b3e99fd26d6d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:02.866347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:02.867319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:02.867699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:02.868063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] PATH_INFO: `/v3/users/14681df64b9c4a6c818ecf6775508122` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:02.869858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14681df64b9c4a6c818ecf6775508122)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:02.870619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1034/2057] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:02 2022] DELETE /identity/v3/users/f1e9ad843d364ae0bac05703288d7041 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:02.873108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e307-7bfd-4311-8132-f3fa48b79478 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:02.873292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:02.873609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e307-7bfd-4311-8132-f3fa48b79478 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:02.874112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e307-7bfd-4311-8132-f3fa48b79478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:02.905833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c539e307-7bfd-4311-8132-f3fa48b79478 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:02.907987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c539e307-7bfd-4311-8132-f3fa48b79478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.020668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1035/2058] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:03.020668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136b01f-7cba-4c5c-b51a-5da6902df0cf None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.020668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136b01f-7cba-4c5c-b51a-5da6902df0cf None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.020668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136b01f-7cba-4c5c-b51a-5da6902df0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.043075 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4136b01f-7cba-4c5c-b51a-5da6902df0cf None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:03.044338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4136b01f-7cba-4c5c-b51a-5da6902df0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.056943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1036/2059] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:03.056943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.056943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.056943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.088729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:03.090641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.101932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1037/2060] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:03.102982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:03.104612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.115673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14681df64b9c4a6c818ecf6775508122'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:03.116869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14681df64b9c4a6c818ecf6775508122'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:03.131106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14681df64b9c4a6c818ecf6775508122'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:03.159668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14681df64b9c4a6c818ecf6775508122'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:03.161479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-62b4b89a-05df-422e-92dc-c0127f29d414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14681df64b9c4a6c818ecf6775508122'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:03.193350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab8152-146d-4020-a3ec-3a5bd4346fd6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:03.197532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:03.198272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.198622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.198910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.201179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.214858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:03.216348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:03.227956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1038/2061] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:03.229487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:03.231785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.254781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986be72-63d2-4bad-b302-536957301646 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:03.263056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:03.264230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.264776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.265279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.266769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:03.267341 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:03.269288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:03.296768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1039/2062] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:03.298677 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:03.306585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1024/2063] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:02 2022] DELETE /identity/v3/users/14681df64b9c4a6c818ecf6775508122 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:03.312255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.312911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.313451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.321058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e28b-8167-4250-8010-9c9406a21d10 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:03.339057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:03.342358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.347090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:03.436946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1025/2064] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:03.448896 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:03.454585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:03.454773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:03.454870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.454870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.454870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.454870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:03.458200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:03.523233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adcf91-e0e6-4013-8920-bdf35e1b3b3f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:03.528384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:03.529080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.529395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.529640 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.541141 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:03.542661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:03.553966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1026/2065] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:28:03.626144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1040/2066] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:03 2022] POST /identity/v3/projects => generated 381 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:03.632630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:03.647035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1506b-5c69-4572-9559-0c744188f841 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:03.792931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:03.794280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.795072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.795830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.797301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:03.801487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:03.952824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1027/2067] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:03 2022] POST /identity/v3/users => generated 291 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:03.959664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:03.983799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb1d75-e662-421f-ad5a-09480cf0dd1b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:03.990594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:03.992218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:03.993803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:03.994436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:03.996336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:03.999604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:04.027084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1041/2068] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:04.030965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:04.056783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f2d2-2b59-423e-a230-a1545534c064 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:04.062846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:04.067843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:04.070584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:04.071281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:04.071465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:04.071999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:04.074373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:04.078412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:04.091347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6dc97-d22b-46a6-8390-26d0d847753a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:04.097763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:04.103761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:04.103761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:04.103761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] PATH_INFO: `/v3/projects/8d2b00583cd54625bda4fa80d98ff191` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:04.103761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2b00583cd54625bda4fa80d98ff191)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:04.104422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:04.108801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1028/2069] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:04.111923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:04.135857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582906c-4834-4495-8062-cb2c586cf7a6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:04.142190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f4d40-d681-4962-8a5f-cb1a400c3a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:04.143229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f4d40-d681-4962-8a5f-cb1a400c3a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:04.144414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f4d40-d681-4962-8a5f-cb1a400c3a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:04.144755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f4d40-d681-4962-8a5f-cb1a400c3a53 admin admin] PATH_INFO: `/v3/projects/7e2078a1ed6a4a17aca8e9dffcc830a2/users/38b3e1b9db4649f683299ce4e964dfed/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:04.168055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f4d40-d681-4962-8a5f-cb1a400c3a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2078a1ed6a4a17aca8e9dffcc830a2, user_id=38b3e1b9db4649f683299ce4e964dfed, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:04.170823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f4d40-d681-4962-8a5f-cb1a400c3a53 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:04.185805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:04.296996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1029/2070] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:04 2022] PUT /identity/v3/projects/7e2078a1ed6a4a17aca8e9dffcc830a2/users/38b3e1b9db4649f683299ce4e964dfed/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:04.306958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2b00583cd54625bda4fa80d98ff191', 'request_id': 'req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:04.319983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:04.320549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:04.321069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:04.325081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2b00583cd54625bda4fa80d98ff191', 'request_id': 'req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:04.369108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] MFA Rules not processed for user `38b3e1b9db4649f683299ce4e964dfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:04.371391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:04.402158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2b00583cd54625bda4fa80d98ff191', 'request_id': 'req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:04.424755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] The token cache is being invalidate because project 8d2b00583cd54625bda4fa80d98ff191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:04.425663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:04.427085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:04.445642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1042/2071] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:04 2022] DELETE /identity/v3/projects/8d2b00583cd54625bda4fa80d98ff191 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:04.484710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1030/2072] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:04.493715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:04.495332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:04.581033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:04.585891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:04.627260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c1bd5-ea3a-49a6-9b0c-78c56029ee02 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:04.728815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:04.729495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:04.729777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:04.730052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:04.730750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:04.732855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:04.745991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1bffc-aebc-483d-9015-d72d2b8a2c01 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:04.754261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:04.755831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:04.756353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:04.756835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] PATH_INFO: `/v3/projects/24a5fd4d3bb44d5d9d89e362c6630882` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:04.758692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a5fd4d3bb44d5d9d89e362c6630882)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:04.762682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:04.859535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1043/2073] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:04 2022] POST /identity/v3/projects => generated 383 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:04.864999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:04.959196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a4037-70d7-46b9-bec0-ddd927ec5580 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:04.971345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a5fd4d3bb44d5d9d89e362c6630882', 'request_id': 'req-392170d3-2cab-418d-9adb-26fb68884e98'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:04.982731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a5fd4d3bb44d5d9d89e362c6630882', 'request_id': 'req-392170d3-2cab-418d-9adb-26fb68884e98'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.044170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.045419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.045892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.046370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.047609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:05.051783 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:05.099262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a5fd4d3bb44d5d9d89e362c6630882', 'request_id': 'req-392170d3-2cab-418d-9adb-26fb68884e98'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.119523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] The token cache is being invalidate because project 24a5fd4d3bb44d5d9d89e362c6630882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:05.120196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.121162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.123126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1031/2074] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:04 2022] DELETE /identity/v3/projects/24a5fd4d3bb44d5d9d89e362c6630882 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:05.128099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:05.130591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:05.155780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392170d3-2cab-418d-9adb-26fb68884e98 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:05.167101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.168539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.169140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.169530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] PATH_INFO: `/v3/users/0d40e7639dcf4965969f96b15ac25527` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.171753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d40e7639dcf4965969f96b15ac25527)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:05.174286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:05.178972 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1044/2075] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:04 2022] POST /identity/v3/users => generated 298 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:05.183203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:05.185261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:05.198807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c5459-a952-43eb-b7e1-397692b5d7c8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:05.210370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.210672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.211330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.212432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.213245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:05.216193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:05.232878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1045/2076] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:05.237305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:05.248802 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f45c0a-d674-4827-94be-2601f2f8caf1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:05.254457 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.255369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.255654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.255931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] PATH_INFO: `/v3/projects/fe2b9aaed5a04b02bad85bf0cfc90f2e/users/1d0bb87ece5c49e4aad7416ec5d67c37/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.282947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2b9aaed5a04b02bad85bf0cfc90f2e, user_id=1d0bb87ece5c49e4aad7416ec5d67c37, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:05.288175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:05.406790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1046/2077] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:05 2022] PUT /identity/v3/projects/fe2b9aaed5a04b02bad85bf0cfc90f2e/users/1d0bb87ece5c49e4aad7416ec5d67c37/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:05.411978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:05.494017 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:05.514135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d40e7639dcf4965969f96b15ac25527'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.514936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d40e7639dcf4965969f96b15ac25527'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.530759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd1417-ec71-48cc-87fb-f6b523e722ad admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:05.533825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d40e7639dcf4965969f96b15ac25527'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.540318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.542051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.542690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.543374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.544918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:05.548452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:05.565537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1047/2078] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:05.568162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d40e7639dcf4965969f96b15ac25527'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.569379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d40e7639dcf4965969f96b15ac25527'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:05.572321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:05.575367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:05.594932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228f2e2-0992-4d35-b3df-2fa2d1a9645b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:05.595992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:05.600746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa56643-6edc-4dc2-b806-5b52fd789b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.601232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa56643-6edc-4dc2-b806-5b52fd789b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.601524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa56643-6edc-4dc2-b806-5b52fd789b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.601815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa56643-6edc-4dc2-b806-5b52fd789b0b admin admin] PATH_INFO: `/v3/projects/fe2b9aaed5a04b02bad85bf0cfc90f2e/users/1d0bb87ece5c49e4aad7416ec5d67c37/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.604018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa56643-6edc-4dc2-b806-5b52fd789b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2b9aaed5a04b02bad85bf0cfc90f2e, user_id=1d0bb87ece5c49e4aad7416ec5d67c37, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:05.606953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa56643-6edc-4dc2-b806-5b52fd789b0b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:05.620570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1048/2079] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:05 2022] PUT /identity/v3/projects/fe2b9aaed5a04b02bad85bf0cfc90f2e/users/1d0bb87ece5c49e4aad7416ec5d67c37/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:05.632142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.632527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.632845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:05.663979 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] MFA Rules not processed for user `1d0bb87ece5c49e4aad7416ec5d67c37`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:05.665461 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:05.698122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1032/2080] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:05 2022] DELETE /identity/v3/users/0d40e7639dcf4965969f96b15ac25527 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:05.771231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1049/2081] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:05.791224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:05.796598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:05.887709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30f3b5-ec85-4355-881e-a028f5927614 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:05.892697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:05.893380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:05.893666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:05.893953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:06.001904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:06.005855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:06.024840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1033/2082] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:06.264158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:06.283260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:06.301480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:06.318732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21649feb-362e-4885-952f-79d651c55a94 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:06.325856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:06.326620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:06.326851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:06.327102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:06.329119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:06.414787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b81e1-423d-40f3-a6b3-5a38f24e5383 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:06.445807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:06.448867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83939e-3c1a-40e1-ac5d-bd31a122dcd4 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:06.464861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1034/2083] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:06.549351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:06.551062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:06.551470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:06.551810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] PATH_INFO: `/v3/projects/730234dfbd1347a3abc4f77fb9ca2f29` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:06.553742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=730234dfbd1347a3abc4f77fb9ca2f29)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:06.556099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:06.726403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '730234dfbd1347a3abc4f77fb9ca2f29', 'request_id': 'req-031b3493-6565-45d0-81b8-66153928a99f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:06.740241 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '730234dfbd1347a3abc4f77fb9ca2f29', 'request_id': 'req-031b3493-6565-45d0-81b8-66153928a99f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:06.835130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '730234dfbd1347a3abc4f77fb9ca2f29', 'request_id': 'req-031b3493-6565-45d0-81b8-66153928a99f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:06.852246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] The token cache is being invalidate because project 730234dfbd1347a3abc4f77fb9ca2f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:06.852630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:06.853430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-031b3493-6565-45d0-81b8-66153928a99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:06.856144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1050/2084] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:06 2022] DELETE /identity/v3/projects/730234dfbd1347a3abc4f77fb9ca2f29 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:06.864136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:06.864478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:06.864768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:06.891072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:06.894096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:06.896126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:06.991894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1035/2085] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:07.000486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.000674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.000942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.027887 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:07.029500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:07.046615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1051/2086] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:07.055957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.072983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844a765-eccb-4e42-9862-70f195708c8e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.080917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.082565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.083041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.083498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.084866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:07.085382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.091349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.111590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1036/2087] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:07.119900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.121891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.122553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:07.139347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ef22b-16b2-4e89-9ceb-3ad846eda3f5 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.142396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d16fda-5fbe-4adc-b34f-dee510f02fea None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.148672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.149404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.149799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.150274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.150723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.150961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.151183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.151602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] PATH_INFO: `/v3/users/38b3e1b9db4649f683299ce4e964dfed` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.152495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.154228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b3e1b9db4649f683299ce4e964dfed)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.156500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.156840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.302267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1037/2088] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:07 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:07.306584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.347303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:07.384397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc279dfd-3b26-4622-a2ef-f4546e47e32d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.390323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.391278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.391661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.392072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.392937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.395501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.508923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1038/2089] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:07 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:07.510566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.517666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b3e1b9db4649f683299ce4e964dfed'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:07.525036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5377de-695c-4ceb-82de-152f833ccd24 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.529379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b3e1b9db4649f683299ce4e964dfed'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:07.530496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b3e1b9db4649f683299ce4e964dfed'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:07.535760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.536635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.537011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.537507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.542454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.542454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.548194 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b3e1b9db4649f683299ce4e964dfed'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:07.549167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ab763d4f-7531-42a0-bbbb-19fe3301a2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b3e1b9db4649f683299ce4e964dfed'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:07.557136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1039/2090] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:07.558418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.560376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:07.573998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976be40-4396-4b1d-bcc4-5ecadd9c0890 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.579856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.579856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.579856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.580352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.580803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.582461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.596632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1040/2091] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:07.602091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.612940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013aa5-d0b8-4a10-bc79-a65d640381e7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.617754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1651a8-b1fa-4363-b3eb-25f97d0a1280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:07.618961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1651a8-b1fa-4363-b3eb-25f97d0a1280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.620123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1651a8-b1fa-4363-b3eb-25f97d0a1280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.620123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1651a8-b1fa-4363-b3eb-25f97d0a1280 admin admin] PATH_INFO: `/v3/projects/79e57723806f4fc493f377fe8d4cf935/users/f43daf84c2944b2a88f9dadc1182620e/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.636758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1651a8-b1fa-4363-b3eb-25f97d0a1280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e57723806f4fc493f377fe8d4cf935, user_id=f43daf84c2944b2a88f9dadc1182620e, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:07.639452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1651a8-b1fa-4363-b3eb-25f97d0a1280 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:07.704743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1052/2092] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:07 2022] DELETE /identity/v3/users/38b3e1b9db4649f683299ce4e964dfed => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:07.711008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.711394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.711575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.731296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:07.733547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:07.801717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1053/2093] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:07.816727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:07.819240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:07.834634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1041/2094] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:07 2022] PUT /identity/v3/projects/79e57723806f4fc493f377fe8d4cf935/users/f43daf84c2944b2a88f9dadc1182620e/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:07.854395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:07.854894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:07.855382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:07.906548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] MFA Rules not processed for user `f43daf84c2944b2a88f9dadc1182620e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:07.908262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:07.920703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102bdac-f6a6-47f1-994f-d8c63dea6037 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:07.994756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1042/2095] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:07.997230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:08.014032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:08.015392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.016084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.021835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.087767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bbaa1-9ec6-456e-83d8-986e349e70ee None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:08.093941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:08.094631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.094912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.095188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.096083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:08.098183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:08.121682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:08.123577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:08.135671 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1054/2096] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:08.216361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1043/2097] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:07 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:08.218164 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:08.230077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f631a1-d96e-4b1c-a56a-2523a30b28d2 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:08.312806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:08.313589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.313977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.314359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.315158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:08.318114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:08.399448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:08.514165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1055/2098] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:08 2022] POST /identity/v3/users => generated 289 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:08.517422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:08.539818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8ca72-1cd6-49a3-a716-34af4e1b9e46 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:08.548717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:08.550964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.551781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.552595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.556696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:08.560411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:08.584126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1044/2099] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:08.585234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:08.605207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45969aad-9891-4b3b-b10a-c8be7dedf93c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:08.623699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd34bf-2f93-4f15-abcf-18e617f86df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:08.625018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd34bf-2f93-4f15-abcf-18e617f86df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.625725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd34bf-2f93-4f15-abcf-18e617f86df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.626825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd34bf-2f93-4f15-abcf-18e617f86df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.627958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd34bf-2f93-4f15-abcf-18e617f86df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:08.632198 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd34bf-2f93-4f15-abcf-18e617f86df2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:08.648818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1056/2100] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:08.654001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:08.669157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a995046-3bda-41dd-a828-5729430cd836 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:08.676183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:08.677282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.677700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.685672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] PATH_INFO: `/v3/projects/16d6ea7e70d44df4aaecd26ee8354e03/users/443593b30c30459090df272915ef4fa8/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.699348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d6ea7e70d44df4aaecd26ee8354e03, user_id=443593b30c30459090df272915ef4fa8, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:08.702274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:08.898539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1045/2101] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:08 2022] PUT /identity/v3/projects/16d6ea7e70d44df4aaecd26ee8354e03/users/443593b30c30459090df272915ef4fa8/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:08.911753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:08.912943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:08.912943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:08.950495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] MFA Rules not processed for user `443593b30c30459090df272915ef4fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:08.952254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:09.022834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.026939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:09.054435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1057/2102] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:09.055416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.160441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ea5c5-14f4-4c85-bed2-673a57237e68 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:09.166231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:09.166990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:09.167459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:09.167878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:09.168591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:09.171139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:09.187978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5c0a0-7ed2-4691-b001-195e0cbcf35f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:09.195482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:09.196785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:09.197314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:09.197693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] PATH_INFO: `/v3/projects/7e2078a1ed6a4a17aca8e9dffcc830a2` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:09.199288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2078a1ed6a4a17aca8e9dffcc830a2)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:09.201578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:09.308739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1058/2103] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:09 2022] POST /identity/v3/projects => generated 377 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:09.315184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.401836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2078a1ed6a4a17aca8e9dffcc830a2', 'request_id': 'req-1036fa7b-a609-43f2-a99e-d72dcd69df64'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.409798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2078a1ed6a4a17aca8e9dffcc830a2', 'request_id': 'req-1036fa7b-a609-43f2-a99e-d72dcd69df64'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.426978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d39e8-1db6-46a2-884c-a6043386825b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:09.525658 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:09.526455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:09.526785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:09.527134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:09.528080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:09.530713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2078a1ed6a4a17aca8e9dffcc830a2', 'request_id': 'req-1036fa7b-a609-43f2-a99e-d72dcd69df64'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.531173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:09.551465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] The token cache is being invalidate because project 7e2078a1ed6a4a17aca8e9dffcc830a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:09.551908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.552635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.554317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1046/2104] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:09 2022] DELETE /identity/v3/projects/7e2078a1ed6a4a17aca8e9dffcc830a2 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:09.558807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.560353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:09.585212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036fa7b-a609-43f2-a99e-d72dcd69df64 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:09.594761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:09.596947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:09.598771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:09.598771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] PATH_INFO: `/v3/users/1d0bb87ece5c49e4aad7416ec5d67c37` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:09.600883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0bb87ece5c49e4aad7416ec5d67c37)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:09.604951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:09.659088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1059/2105] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:09 2022] POST /identity/v3/users => generated 295 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:09.664104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.666136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:09.681904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1eea-d156-4064-b254-fe663c3c0962 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:09.689193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:09.691275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:09.691969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:09.692258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:09.692598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:09.693359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:09.695025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:09.706410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1060/2106] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:09.712987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.725077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca74e-8656-4dba-8598-e894b01fb940 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:09.730760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:09.731711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:09.731992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:09.732263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] PATH_INFO: `/v3/projects/8150a60bcff54883bfd6bf31d84a19eb/users/7b534b6a7f644d6383c0696ac7d6f074/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:09.752769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8150a60bcff54883bfd6bf31d84a19eb, user_id=7b534b6a7f644d6383c0696ac7d6f074, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:09.755041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:09.755547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:09.879701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1061/2107] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:09 2022] PUT /identity/v3/projects/8150a60bcff54883bfd6bf31d84a19eb/users/7b534b6a7f644d6383c0696ac7d6f074/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:09.885610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:09.890858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0bb87ece5c49e4aad7416ec5d67c37'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.891924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0bb87ece5c49e4aad7416ec5d67c37'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.903316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0bb87ece5c49e4aad7416ec5d67c37'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.922905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0bb87ece5c49e4aad7416ec5d67c37'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.923856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-66a0bba6-5d5f-4d1b-aa92-7df78a4ddddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0bb87ece5c49e4aad7416ec5d67c37'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:09.994478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14823d1-a29d-45d4-b7a0-ded1756b1d4f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:10.001697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:10.002702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:10.003264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:10.003733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:10.004821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:10.007354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:10.022570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1062/2108] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:10.027324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:10.029482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:10.043294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1047/2109] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:09 2022] DELETE /identity/v3/users/1d0bb87ece5c49e4aad7416ec5d67c37 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:10.051597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8ab59-d130-407e-ad73-e377c43dee6f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:10.113197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:10.114447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:10.114732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:10.115019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] PATH_INFO: `/v3/projects/8150a60bcff54883bfd6bf31d84a19eb/users/7b534b6a7f644d6383c0696ac7d6f074/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:10.117380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8150a60bcff54883bfd6bf31d84a19eb, user_id=7b534b6a7f644d6383c0696ac7d6f074, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:10.120444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:10.132625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1063/2110] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:10 2022] PUT /identity/v3/projects/8150a60bcff54883bfd6bf31d84a19eb/users/7b534b6a7f644d6383c0696ac7d6f074/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:10.144436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:10.145047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:10.145661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:10.181214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] MFA Rules not processed for user `7b534b6a7f644d6383c0696ac7d6f074`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:10.183717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:10.277505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1048/2111] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:10.294160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:10.297672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:10.336638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:10.408233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd7dc-3b74-4366-bcfe-764551dbc634 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:10.415302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:10.416565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:10.417116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:10.417630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:10.437140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:10.439981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:10.463001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1064/2112] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:10.651296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:10.671434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6da64-071e-4509-ad4b-043cf718528e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:10.678019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:10.679148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:10.679701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:10.680074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:10.683082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:10.742053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:10.744794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:10.783835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:10.786798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae93c8-efe6-45c9-9621-ebb5be4d4692 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:10.802084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1049/2113] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:10.857319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243ede4-09dc-4f6d-ab02-3f85d7e84597 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:10.863339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:10.864506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:10.864947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:10.865431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] PATH_INFO: `/v3/projects/fe2b9aaed5a04b02bad85bf0cfc90f2e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:10.867443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2b9aaed5a04b02bad85bf0cfc90f2e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:10.870781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:10.913108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:11.029495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2b9aaed5a04b02bad85bf0cfc90f2e', 'request_id': 'req-35e66b30-dba6-484e-896c-9617787135a4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:11.037044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2b9aaed5a04b02bad85bf0cfc90f2e', 'request_id': 'req-35e66b30-dba6-484e-896c-9617787135a4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:11.126302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2b9aaed5a04b02bad85bf0cfc90f2e', 'request_id': 'req-35e66b30-dba6-484e-896c-9617787135a4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:11.142616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] The token cache is being invalidate because project fe2b9aaed5a04b02bad85bf0cfc90f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:11.143006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:11.144006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-35e66b30-dba6-484e-896c-9617787135a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:11.145896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1065/2114] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:10 2022] DELETE /identity/v3/projects/fe2b9aaed5a04b02bad85bf0cfc90f2e => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:11.152393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.152948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.153655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.185553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:11.186971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:11.270450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:11.285370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1050/2115] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:11.295700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.296227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.296776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.322035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:11.323480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:11.332806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1066/2116] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:11.337905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:11.349491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945a4e8-067d-4fd0-9b51-bab244a1cb27 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:11.354850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:11.355681 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.356025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.356424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.357313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:11.357662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:11.359529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:11.374359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1051/2117] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:11.379711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:11.390361 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df5e2e-8687-4a9b-b2f4-00747ce0b89d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:11.396375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:11.397069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.397354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.397721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.398428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:11.400482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:11.447348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:11.578070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1067/2118] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:11 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:11.579980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:11.595357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f061b00-d46f-43b9-a348-2f7bb4bfe081 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:11.703073 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:11.704245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.704732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.705223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.706266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:11.709919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:11.824404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1052/2119] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:11 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:11.833323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:11.847542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fbfe8-826b-4c3f-a8cc-2217f1f4f335 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:11.853844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:11.854540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.854825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.855111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.855864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:11.857707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:11.872166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1068/2120] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:11.879732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:11.902504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeb240-ece3-4b8b-90ce-1fd05a63a512 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:11.908019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd9835-6c51-4b30-86cb-0b264885eb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:11.910201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd9835-6c51-4b30-86cb-0b264885eb09 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.910978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd9835-6c51-4b30-86cb-0b264885eb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.911707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd9835-6c51-4b30-86cb-0b264885eb09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:11.913382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd9835-6c51-4b30-86cb-0b264885eb09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:11.917252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd9835-6c51-4b30-86cb-0b264885eb09 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:11.939341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1053/2121] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:11.944925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:11.962434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b80cc0-f151-42d8-8f7a-b3f755b59489 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:11.970722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:11.973183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:11.973770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:11.974312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] PATH_INFO: `/v3/projects/60982dfdc53d4948b6fb4033f58717d5/users/931f36f5a94941ca8a634313c2b3574b/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:12.004322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60982dfdc53d4948b6fb4033f58717d5, user_id=931f36f5a94941ca8a634313c2b3574b, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:12.009909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:12.144845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1069/2122] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:11 2022] PUT /identity/v3/projects/60982dfdc53d4948b6fb4033f58717d5/users/931f36f5a94941ca8a634313c2b3574b/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:12.156955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:12.157379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:12.157769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:12.197201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] MFA Rules not processed for user `931f36f5a94941ca8a634313c2b3574b`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:12.198761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:12.278028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1054/2123] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:12.298171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:12.301486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:12.366927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ab882-c16a-46f9-b627-f9136e830195 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:12.371164 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6d9a5-7c58-4051-84d3-dff77f1b0a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:12.371836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6d9a5-7c58-4051-84d3-dff77f1b0a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:12.372111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6d9a5-7c58-4051-84d3-dff77f1b0a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:12.372436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6d9a5-7c58-4051-84d3-dff77f1b0a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:12.382763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6d9a5-7c58-4051-84d3-dff77f1b0a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:12.384576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6d9a5-7c58-4051-84d3-dff77f1b0a6e service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:12.398483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1070/2124] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:12.429432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:12.516137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4fd34-b80c-420c-899c-7badf51b617c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:12.521521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:12.522461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:12.522743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:12.523020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] PATH_INFO: `/v3/users/931f36f5a94941ca8a634313c2b3574b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:12.530605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=931f36f5a94941ca8a634313c2b3574b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:12.530605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:12.776032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '931f36f5a94941ca8a634313c2b3574b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:12.776903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '931f36f5a94941ca8a634313c2b3574b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:12.786571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '931f36f5a94941ca8a634313c2b3574b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:12.801991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '931f36f5a94941ca8a634313c2b3574b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:12.803075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '931f36f5a94941ca8a634313c2b3574b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:12.928738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1055/2125] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:12 2022] DELETE /identity/v3/users/931f36f5a94941ca8a634313c2b3574b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:12.936358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:12.937782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:12.938302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:12.976184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:12.978325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:13.077962 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1071/2126] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:13.091754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:13.093460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:13.179826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcb7be-fac7-4569-9741-7cc7ae2db3d1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:13.186538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35a846-9d0d-4fa1-8260-486815f234c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:13.187642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35a846-9d0d-4fa1-8260-486815f234c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:13.188162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35a846-9d0d-4fa1-8260-486815f234c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:13.188718 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35a846-9d0d-4fa1-8260-486815f234c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:13.206907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35a846-9d0d-4fa1-8260-486815f234c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:13.209894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35a846-9d0d-4fa1-8260-486815f234c0 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:13.228512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1056/2127] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:28:13.743226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:13.825820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:13.827777 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:13.842202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea350e0-b8af-462a-a6fd-24d022624154 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:13.848385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:13.848761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:13.849069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:13.849407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] PATH_INFO: `/v3/projects/60982dfdc53d4948b6fb4033f58717d5` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:13.851909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=60982dfdc53d4948b6fb4033f58717d5)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:13.853940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:13.855535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:14.014428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60982dfdc53d4948b6fb4033f58717d5', 'request_id': 'req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:14.027205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60982dfdc53d4948b6fb4033f58717d5', 'request_id': 'req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:14.172925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60982dfdc53d4948b6fb4033f58717d5', 'request_id': 'req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:14.197083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] The token cache is being invalidate because project 60982dfdc53d4948b6fb4033f58717d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:14.197589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:14.204544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-6edb944e-55ba-4c5b-88c0-ce428e5a84fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:14.218991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1072/2128] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:13 2022] DELETE /identity/v3/projects/60982dfdc53d4948b6fb4033f58717d5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:14.222818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.223393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.223898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.285680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:14.287819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:14.408082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1057/2129] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:14.417657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.418037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.418362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.439043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:14.440946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:14.453337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1073/2130] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:14.458169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:14.468774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595847-0544-4caa-a1f3-6c77a5b151c7 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:14.474829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:14.475512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.475799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.476089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.476970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:14.477276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:14.479010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:14.493852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1058/2131] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:14.498903 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:14.510292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d35a-bec0-41cc-b272-3bfc5eb1bdc7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:14.515774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:14.516710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.517043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.517321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.518930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:14.521329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:14.642675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1074/2132] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:14 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:14.647356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:14.658266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27724-7ab6-4ce8-ab60-f3dd503ef302 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:14.756498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:14.757620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.758094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.758563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.759786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:14.763416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:14.886527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1059/2133] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:14 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:14.891878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:14.903660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fabb7-5383-4a13-a79d-4a7ee006b3de admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:14.909191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:14.909938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.910221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.910502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.911261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:14.913025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:14.928753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1075/2134] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:14.930126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:14.941922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32334a3c-2ae1-4daf-9001-fbfc4fe3a621 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:14.947224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8b80b-0b85-42ed-926b-c68ed69c2f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:14.947914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8b80b-0b85-42ed-926b-c68ed69c2f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.949168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8b80b-0b85-42ed-926b-c68ed69c2f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.949168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8b80b-0b85-42ed-926b-c68ed69c2f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:14.949373 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8b80b-0b85-42ed-926b-c68ed69c2f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:14.951063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8b80b-0b85-42ed-926b-c68ed69c2f29 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:14.962655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1060/2135] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:28:14.967577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:14.978985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa745-8eba-41e3-a2f5-01a9bccc6b47 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:14.985201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:14.986138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:14.986482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:14.986830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] PATH_INFO: `/v3/projects/a303a1a729ee4351bbedb7e8eb2a9339/users/46f759e116564db4a3c769dba420ca00/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:15.007737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a303a1a729ee4351bbedb7e8eb2a9339, user_id=46f759e116564db4a3c769dba420ca00, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:15.011527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:15.135747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1076/2136] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:14 2022] PUT /identity/v3/projects/a303a1a729ee4351bbedb7e8eb2a9339/users/46f759e116564db4a3c769dba420ca00/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:15.172716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:15.173068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:15.173446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:15.205460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] MFA Rules not processed for user `46f759e116564db4a3c769dba420ca00`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:15.206767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:15.285953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1061/2137] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:15.313185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:15.313185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:15.419256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdac69-0bd7-48c5-9405-dae1086a7322 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:15.426434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79200f86-2564-4bc3-aece-38d9163831bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:15.427931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200f86-2564-4bc3-aece-38d9163831bb service glance] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:15.428500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200f86-2564-4bc3-aece-38d9163831bb service glance] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:15.429173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200f86-2564-4bc3-aece-38d9163831bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:15.453058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79200f86-2564-4bc3-aece-38d9163831bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:15.456674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79200f86-2564-4bc3-aece-38d9163831bb service glance] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:15.484810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1077/2138] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:16.031716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:16.148817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:17.355327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:17.357585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:17.471221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251f2e8-c006-4943-98ab-ef4e1a520c3f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:17.476449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:17.477416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:17.477703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:17.477980 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] PATH_INFO: `/v3/users/f43daf84c2944b2a88f9dadc1182620e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:17.479267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43daf84c2944b2a88f9dadc1182620e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:17.481837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:17.826414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43daf84c2944b2a88f9dadc1182620e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:17.827521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43daf84c2944b2a88f9dadc1182620e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:17.850521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43daf84c2944b2a88f9dadc1182620e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:17.857359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43daf84c2944b2a88f9dadc1182620e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:17.858111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43daf84c2944b2a88f9dadc1182620e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:17.910267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:17.993975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1062/2139] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:17 2022] DELETE /identity/v3/users/f43daf84c2944b2a88f9dadc1182620e => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:17.999813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:18.000838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:18.003092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:18.029130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:18.031256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:18.120050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1078/2140] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:18.133680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:18.135534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:18.215756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8c0cf-7ce3-4179-ab2f-75c4b1885172 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:18.220192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:18.221033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:18.221410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:18.221753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:18.235169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:18.238502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:18.261297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1063/2141] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:28:18.579216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:18.932467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:18.932467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:18.942739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91a678-4206-4300-8319-b4eb0847f4d0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:18.949809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:18.950888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:18.951355 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:18.951921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] PATH_INFO: `/v3/projects/79e57723806f4fc493f377fe8d4cf935` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:18.953699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e57723806f4fc493f377fe8d4cf935)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:18.957173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:19.520975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e57723806f4fc493f377fe8d4cf935', 'request_id': 'req-9d6613c2-2efc-4884-95d0-42aea8873aec'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:19.532612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e57723806f4fc493f377fe8d4cf935', 'request_id': 'req-9d6613c2-2efc-4884-95d0-42aea8873aec'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:19.645547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e57723806f4fc493f377fe8d4cf935', 'request_id': 'req-9d6613c2-2efc-4884-95d0-42aea8873aec'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:19.669186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] The token cache is being invalidate because project 79e57723806f4fc493f377fe8d4cf935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:19.669664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:19.670815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:19.673268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1079/2142] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:18 2022] DELETE /identity/v3/projects/79e57723806f4fc493f377fe8d4cf935 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:19.684161 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:19.706218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69af84c-010b-450c-8760-d38bbe6f22a7 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:19.795145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:19.796054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:19.796374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:19.796655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] PATH_INFO: `/v3/users/443593b30c30459090df272915ef4fa8` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:19.797986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=443593b30c30459090df272915ef4fa8)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:19.800891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:19.806462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:20.086778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443593b30c30459090df272915ef4fa8'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:20.088082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443593b30c30459090df272915ef4fa8'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:20.100930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443593b30c30459090df272915ef4fa8'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:20.117224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443593b30c30459090df272915ef4fa8'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:20.118047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443593b30c30459090df272915ef4fa8'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:20.263146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1064/2143] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:19 2022] DELETE /identity/v3/users/443593b30c30459090df272915ef4fa8 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:20.710983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:20.715211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:20.745571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6613c2-2efc-4884-95d0-42aea8873aec admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:20.847353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:20.848430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:20.848907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:20.849464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] PATH_INFO: `/v3/users/8c23233891e84931867c27b5ebebfb46` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:20.851254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c23233891e84931867c27b5ebebfb46)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:20.853473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:20.962843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:20.965372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:20.988967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223d0cf-8be6-43e0-ba14-52d2d931da33 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:21.004934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:21.004934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:21.005906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:21.006713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] PATH_INFO: `/v3/projects/16d6ea7e70d44df4aaecd26ee8354e03` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:21.009555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d6ea7e70d44df4aaecd26ee8354e03)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:21.012899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:21.127591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:21.135158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c23233891e84931867c27b5ebebfb46'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.141571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d6ea7e70d44df4aaecd26ee8354e03', 'request_id': 'req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.150072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d6ea7e70d44df4aaecd26ee8354e03', 'request_id': 'req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.154148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c23233891e84931867c27b5ebebfb46'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.155114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c23233891e84931867c27b5ebebfb46'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.173462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c23233891e84931867c27b5ebebfb46'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.174333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1014d6bc-ad1f-44c3-8a05-6001b057e5df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c23233891e84931867c27b5ebebfb46'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.222501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d6ea7e70d44df4aaecd26ee8354e03', 'request_id': 'req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.236135 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] The token cache is being invalidate because project 16d6ea7e70d44df4aaecd26ee8354e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:21.236479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.237200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.239041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1065/2144] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:20 2022] DELETE /identity/v3/projects/16d6ea7e70d44df4aaecd26ee8354e03 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:21.246348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:21.247920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:21.317126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1080/2145] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:20 2022] DELETE /identity/v3/users/8c23233891e84931867c27b5ebebfb46 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:21.324935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:21.325368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:21.325745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:21.327415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a2deb-ab69-4e8d-bbf4-40e7d05f435e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:21.344974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:21.346441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:21.400366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:21.401651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:21.401998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:21.402341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] PATH_INFO: `/v3/users/7b534b6a7f644d6383c0696ac7d6f074` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:21.403802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b534b6a7f644d6383c0696ac7d6f074)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:21.406067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:21.418317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1081/2146] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:21.431057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:21.433059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:21.509871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08fdc-f4c2-4206-a94a-9bd78d329a0a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:21.514025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:21.514705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:21.514990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:21.515271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:21.528970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:21.531442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:21.544811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1082/2147] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:21.643346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:21.794514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b534b6a7f644d6383c0696ac7d6f074'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.795839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b534b6a7f644d6383c0696ac7d6f074'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.819259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b534b6a7f644d6383c0696ac7d6f074'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.855035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b534b6a7f644d6383c0696ac7d6f074'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.857074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b534b6a7f644d6383c0696ac7d6f074'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:21.879456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:21.983023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1066/2148] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:21 2022] DELETE /identity/v3/users/7b534b6a7f644d6383c0696ac7d6f074 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:22.292842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:22.295473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:22.315721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:22.357785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1a5eb-4370-4dbd-8831-a09421a8972f service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:22.460787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:22.461716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:22.462001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:22.462284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] PATH_INFO: `/v3/projects/0ab2e5a8f8c1436289b519d108694734` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:22.464383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab2e5a8f8c1436289b519d108694734)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:22.466408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:22.663219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab2e5a8f8c1436289b519d108694734', 'request_id': 'req-287712f6-4730-4695-ad78-ccef9f303f14'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:22.681435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab2e5a8f8c1436289b519d108694734', 'request_id': 'req-287712f6-4730-4695-ad78-ccef9f303f14'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:22.778221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab2e5a8f8c1436289b519d108694734', 'request_id': 'req-287712f6-4730-4695-ad78-ccef9f303f14'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:22.809756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] The token cache is being invalidate because project 0ab2e5a8f8c1436289b519d108694734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:22.810269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:22.811455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-287712f6-4730-4695-ad78-ccef9f303f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:22.819540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:22.824350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1083/2149] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:22 2022] DELETE /identity/v3/projects/0ab2e5a8f8c1436289b519d108694734 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:22.989569 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:22.995400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:23.046207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f5e71-eaf3-4a0f-a282-8797af389394 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:23.122574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:23.123376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:23.123662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:23.123945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] PATH_INFO: `/v3/projects/8150a60bcff54883bfd6bf31d84a19eb` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:23.125333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8150a60bcff54883bfd6bf31d84a19eb)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:23.128034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:23.262228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8150a60bcff54883bfd6bf31d84a19eb', 'request_id': 'req-95a56883-94a3-4e5e-8541-88acdbd19038'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:23.267410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:23.273037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8150a60bcff54883bfd6bf31d84a19eb', 'request_id': 'req-95a56883-94a3-4e5e-8541-88acdbd19038'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:23.387843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8150a60bcff54883bfd6bf31d84a19eb', 'request_id': 'req-95a56883-94a3-4e5e-8541-88acdbd19038'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:23.406744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] The token cache is being invalidate because project 8150a60bcff54883bfd6bf31d84a19eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:23.407198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:23.408820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-95a56883-94a3-4e5e-8541-88acdbd19038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:23.410489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1067/2150] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:22 2022] DELETE /identity/v3/projects/8150a60bcff54883bfd6bf31d84a19eb => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:23.417508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:23.418108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:23.418108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:23.458103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:23.460217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:23.583512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1084/2151] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:23.587970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:23.588443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:23.588823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:23.617134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:23.618416 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:23.632123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1068/2152] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:23.636833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:23.652638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b63ac-7bb2-432e-b6a9-5723ec0a1dce None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:23.658067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:23.658772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:23.659055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:23.659334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:23.660110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:23.660544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:23.662084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:23.691718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1085/2153] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:23.691962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:23.705509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c9f06-4df1-4de6-bdcb-07b58b0c92bd None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:23.711274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:23.711999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:23.712511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:23.712798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:23.713559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:23.716350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:23.745483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:23.850231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1069/2154] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:23 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:23.854678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:23.867059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec37c2-0784-424e-9cd9-ad235757b9cc admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:23.954301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:23.954301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:23.954301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:23.954301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:23.955880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:23.959941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:24.077705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1086/2155] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:23 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:28:24.082391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:24.093714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba229-31d8-434b-9d1c-ab97736813b3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:24.101578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:24.104041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:24.105276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:24.106346 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:24.108511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:24.114174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:24.131663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1070/2156] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:24.136810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:24.151988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae6be-0782-476f-b33e-8bf74f8675f4 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:24.157952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03456-70af-4c29-8cb3-923ea1624fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:24.158633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03456-70af-4c29-8cb3-923ea1624fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:24.158919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03456-70af-4c29-8cb3-923ea1624fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:24.159197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03456-70af-4c29-8cb3-923ea1624fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:24.160129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e03456-70af-4c29-8cb3-923ea1624fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:24.161934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e03456-70af-4c29-8cb3-923ea1624fc5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:24.173493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1087/2157] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:24.178169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:24.192429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4426156-18e1-4ff2-a646-72a847b3bbb2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:24.197978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:24.198894 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:24.199178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:24.199466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] PATH_INFO: `/v3/projects/e28426853fdb42f08d50aad76ff2e7a7/users/51166e77ba0d44ca8119947228dd1cf0/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:24.217211 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28426853fdb42f08d50aad76ff2e7a7, user_id=51166e77ba0d44ca8119947228dd1cf0, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:24.220312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:24.273655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:24.345950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1071/2158] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:24 2022] PUT /identity/v3/projects/e28426853fdb42f08d50aad76ff2e7a7/users/51166e77ba0d44ca8119947228dd1cf0/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:24.359159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91d097-0a24-49e2-8651-053942b7f09d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:24.359452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91d097-0a24-49e2-8651-053942b7f09d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:24.359740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91d097-0a24-49e2-8651-053942b7f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:24.391223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ff91d097-0a24-49e2-8651-053942b7f09d None None] MFA Rules not processed for user `51166e77ba0d44ca8119947228dd1cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:24.392626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ff91d097-0a24-49e2-8651-053942b7f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:24.466312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1088/2159] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:24.479703 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:24.481619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:24.587225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed321d1-5c98-4570-9657-8088e94fd4c8 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:24.591397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:24.592109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:24.592477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:24.592797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:24.603874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:24.605603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:24.616420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1072/2160] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:28:24.787678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:25.381917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:25.838718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:26.200821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:26.249495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:26.759729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:26.988709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:27.416161 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:27.597084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:27.690748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:28.255339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:28.436499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:28.653584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:28.728052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:28.967846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:29.007738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:29.190577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:29.255780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:29.432323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:29.558114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:29.590345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.061993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.090450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.331256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.370736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.525050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.555224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.728074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.758144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.924352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:30.952311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:31.274317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:31.386853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:31.918524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:32.464736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:32.581114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:33.187172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:33.346110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:33.456257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:33.774839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:40.728266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:40.728990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:40.729169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:40.770672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:40.772778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:40.812211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:40.813561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:40.823182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:40.850337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:40.874099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2337-7866-4042-ac3d-f4f0cacff345 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:40.890672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:40.895148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:40.895605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:40.895895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] PATH_INFO: `/v3/users/46f759e116564db4a3c769dba420ca00` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:40.899191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f759e116564db4a3c769dba420ca00)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:40.901639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:40.935046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1089/2161] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:28:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:41.002969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:41.004942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:41.156140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202fd91-3faa-42af-b12a-d160d75ce9c0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:41.161753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ae9e8-826d-4209-b832-94292d3907b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:41.162768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae9e8-826d-4209-b832-94292d3907b4 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:41.162768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae9e8-826d-4209-b832-94292d3907b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:41.162991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae9e8-826d-4209-b832-94292d3907b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:41.173318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ae9e8-826d-4209-b832-94292d3907b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:41.175026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ae9e8-826d-4209-b832-94292d3907b4 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:41.179390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f759e116564db4a3c769dba420ca00'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:41.182076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f759e116564db4a3c769dba420ca00'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:41.185563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1090/2162] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:28:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:28:41.187429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8c095-518d-49d7-8622-b0b3922d1511 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:41.188208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8c095-518d-49d7-8622-b0b3922d1511 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:41.188381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8c095-518d-49d7-8622-b0b3922d1511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:41.207021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f759e116564db4a3c769dba420ca00'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:41.213253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-99b8c095-518d-49d7-8622-b0b3922d1511 None None] MFA Rules not processed for user `34a7ef6d33b141ce8fb1cec517faffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:41.213253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-99b8c095-518d-49d7-8622-b0b3922d1511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:41.245800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f759e116564db4a3c769dba420ca00'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:41.252524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-925b63d5-adce-49c2-8db9-2ede15c6c338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f759e116564db4a3c769dba420ca00'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:41.396364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1091/2163] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:41.418522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1073/2164] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:40 2022] DELETE /identity/v3/users/46f759e116564db4a3c769dba420ca00 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:41.423075 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:41.423435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:41.423739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:41.447608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:41.448901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:41.534026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:41.534189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:41.534487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:41.565308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] MFA Rules not processed for user `aeffa44fe38a4204ae21b52fe827cedc`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:41.569390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:41.593399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:41.599206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1092/2165] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:41.615009 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:41.617018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:41.699597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1074/2166] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:41.772869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92edf5-1ef0-4b1d-9054-c4cfa244177e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:41.784884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7086623-a152-4a87-893c-44d18140ed9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:41.785570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7086623-a152-4a87-893c-44d18140ed9d service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:41.786022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7086623-a152-4a87-893c-44d18140ed9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:41.786571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7086623-a152-4a87-893c-44d18140ed9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:41.811609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7086623-a152-4a87-893c-44d18140ed9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:41.813683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7086623-a152-4a87-893c-44d18140ed9d service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:41.853447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1093/2167] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:28:42.605531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:42.607104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:42.635883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7c76c-e20a-4466-9498-a1fcee39f8be None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:42.653036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:42.654116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:42.654507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:42.654888 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] PATH_INFO: `/v3/projects/a303a1a729ee4351bbedb7e8eb2a9339` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:42.657581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a303a1a729ee4351bbedb7e8eb2a9339)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:42.663226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:42.880551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a303a1a729ee4351bbedb7e8eb2a9339', 'request_id': 'req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:42.882012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:42.896146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a303a1a729ee4351bbedb7e8eb2a9339', 'request_id': 'req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:43.042539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a303a1a729ee4351bbedb7e8eb2a9339', 'request_id': 'req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:43.080533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] The token cache is being invalidate because project a303a1a729ee4351bbedb7e8eb2a9339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:28:43.081328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:43.082455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b90b7c33-1cb8-44de-8f1d-8086dc22f20d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:43.086555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1075/2168] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:28:42 2022] DELETE /identity/v3/projects/a303a1a729ee4351bbedb7e8eb2a9339 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:28:43.093969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:43.094794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:43.094938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:43.130093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:43.131340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:43.224789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1094/2169] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:43.238607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:43.239395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:43.239877 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:43.272725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:43.275352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:43.304499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1076/2170] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:43.305978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:43.319720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf1e08-6f93-4e62-b771-0827b7b6ae96 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:43.325868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:43.326848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:43.327701 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:43.327991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:43.329119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:28:43.329441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:43.331986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:43.362023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1095/2171] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:28:43.364794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:43.380828 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b3d5-2857-4c8d-ba8b-2dd205fd4e1a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:43.393554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:43.395399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:43.396314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:43.397196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:43.399342 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:43.404191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:43.638863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1077/2172] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:28:43 2022] POST /identity/v3/projects => generated 399 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:43.640504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:43.652137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1bd77-a702-42d4-816e-c490ffd5eb9c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:43.875657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:43.876257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:43.876590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:43.876868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:43.877612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:43.879653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:43.922924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:44.079396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1096/2173] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:28:43 2022] POST /identity/v3/users => generated 300 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:28:44.084390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:44.138776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1688b0-5a36-4721-b73b-88d4b57c8376 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:44.156860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:44.158329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:44.159292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:44.160055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:44.161930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:44.167451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:44.202488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1078/2174] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:44.208814 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:44.235202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cca46-c01b-40b4-b0ee-e7acaa1aa692 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:44.262505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4bbd2-57d8-4abe-86d7-91d7cebd212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:44.262505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4bbd2-57d8-4abe-86d7-91d7cebd212b admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:44.262505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4bbd2-57d8-4abe-86d7-91d7cebd212b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:44.262505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4bbd2-57d8-4abe-86d7-91d7cebd212b admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:44.263153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4bbd2-57d8-4abe-86d7-91d7cebd212b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:44.265010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4bbd2-57d8-4abe-86d7-91d7cebd212b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:44.294929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1097/2175] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:28:44.306481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:44.332790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7d468-c546-4f35-9025-c2530c22cbd2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:44.380149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:44.382618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:44.386687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:44.436752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] PATH_INFO: `/v3/projects/3bf577b226ba4c878283ec76c34d0da3/users/f0ad1343e91d45f28fc6a1e714577eab/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:44.436752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf577b226ba4c878283ec76c34d0da3, user_id=f0ad1343e91d45f28fc6a1e714577eab, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:44.436752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:44.770509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1079/2176] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:28:44 2022] PUT /identity/v3/projects/3bf577b226ba4c878283ec76c34d0da3/users/f0ad1343e91d45f28fc6a1e714577eab/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:44.784368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:44.785010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:44.786072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:44.826469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] MFA Rules not processed for user `f0ad1343e91d45f28fc6a1e714577eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:44.829548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:44.979598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1098/2177] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:28:45.061950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:45.067377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:45.261565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0aa24-eb9c-4811-9f55-7826f7969076 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:45.270311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f3f1e-b9df-487f-9152-5c4da8803b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:45.271410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f3f1e-b9df-487f-9152-5c4da8803b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:45.271907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f3f1e-b9df-487f-9152-5c4da8803b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:45.276241 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f3f1e-b9df-487f-9152-5c4da8803b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:45.290293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f3f1e-b9df-487f-9152-5c4da8803b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:45.292362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f3f1e-b9df-487f-9152-5c4da8803b83 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:45.307549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1080/2178] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:28:54.975579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:55.304043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=59,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:28:58.587123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:58.689586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd059634-d263-49e0-89a9-e66727299b1d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:58.697728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:58.697728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:58.697728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:58.697728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] PATH_INFO: `/v3/users/f0ad1343e91d45f28fc6a1e714577eab` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:58.700185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0ad1343e91d45f28fc6a1e714577eab)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:58.702087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:58.964603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0ad1343e91d45f28fc6a1e714577eab'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:58.981956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0ad1343e91d45f28fc6a1e714577eab'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:58.983068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0ad1343e91d45f28fc6a1e714577eab'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:59.008197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0ad1343e91d45f28fc6a1e714577eab'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:59.011554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0ad1343e91d45f28fc6a1e714577eab'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:28:59.115382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1099/2179] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:28:58 2022] DELETE /identity/v3/users/f0ad1343e91d45f28fc6a1e714577eab => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:28:59.121724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:59.123144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:59.123313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:59.150165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:28:59.152105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:59.243808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1081/2180] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:28:59.259522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:28:59.272339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:28:59.408651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe16a6-4c4c-440f-9cb6-958d4d21b5f7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:28:59.408651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:28:59.411865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:28:59.411865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:28:59.411865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:28:59.430342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:28:59.432836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:28:59.446504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1100/2181] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:29:00.103418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:00.106818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:00.141054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c0948-7ae4-4f2b-adc9-20f7d523b64c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:00.171093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:00.171093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:00.172380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:00.172380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] PATH_INFO: `/v3/projects/3bf577b226ba4c878283ec76c34d0da3` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:00.178053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf577b226ba4c878283ec76c34d0da3)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:00.182934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:00.358605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf577b226ba4c878283ec76c34d0da3', 'request_id': 'req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:00.367523 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf577b226ba4c878283ec76c34d0da3', 'request_id': 'req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:00.514565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf577b226ba4c878283ec76c34d0da3', 'request_id': 'req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:00.535383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] The token cache is being invalidate because project 3bf577b226ba4c878283ec76c34d0da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:29:00.535953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:00.537367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9c9df28c-b52e-4f71-8b8e-fc4b1bf7cfd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:00.554163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1082/2182] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:29:00 2022] DELETE /identity/v3/projects/3bf577b226ba4c878283ec76c34d0da3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:29:00.655655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:00.659213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:00.688174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc47b-2ac1-46e0-a248-ebaa665b87af service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:00.731254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:00.825272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:00.827751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:00.828262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:00.828798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] PATH_INFO: `/v3/users/51166e77ba0d44ca8119947228dd1cf0` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:00.831201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51166e77ba0d44ca8119947228dd1cf0)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:00.835177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:01.092733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51166e77ba0d44ca8119947228dd1cf0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:01.107148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51166e77ba0d44ca8119947228dd1cf0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:01.108374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51166e77ba0d44ca8119947228dd1cf0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:01.132700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51166e77ba0d44ca8119947228dd1cf0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:01.133805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51166e77ba0d44ca8119947228dd1cf0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:01.271386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1101/2183] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:29:00 2022] DELETE /identity/v3/users/51166e77ba0d44ca8119947228dd1cf0 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:29:01.277119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:01.277631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:01.277756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:01.300674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:01.302508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:01.373550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1083/2184] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:01.384648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:01.386095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:01.457503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c790f0-0879-409a-8e04-f2cd0f1d632f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:01.461965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648ac98-420c-4e96-86ea-23c7c4c676b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:01.462763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ac98-420c-4e96-86ea-23c7c4c676b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:01.463092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ac98-420c-4e96-86ea-23c7c4c676b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:01.463422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ac98-420c-4e96-86ea-23c7c4c676b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:01.474454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648ac98-420c-4e96-86ea-23c7c4c676b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:01.475998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648ac98-420c-4e96-86ea-23c7c4c676b9 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:01.487470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1102/2185] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:29:02.365419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:02.365419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:02.387391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe40c89-bb84-4349-8586-3b8ac54ceaef None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:02.395716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:02.397382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:02.397876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:02.398378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] PATH_INFO: `/v3/projects/e28426853fdb42f08d50aad76ff2e7a7` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:02.401282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28426853fdb42f08d50aad76ff2e7a7)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:02.406320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:02.444537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:02.592395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28426853fdb42f08d50aad76ff2e7a7', 'request_id': 'req-982f2d19-4d79-45dd-94fe-94567d68cedd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:02.600818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28426853fdb42f08d50aad76ff2e7a7', 'request_id': 'req-982f2d19-4d79-45dd-94fe-94567d68cedd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:02.724315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28426853fdb42f08d50aad76ff2e7a7', 'request_id': 'req-982f2d19-4d79-45dd-94fe-94567d68cedd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:02.739311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] The token cache is being invalidate because project e28426853fdb42f08d50aad76ff2e7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:29:02.739768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:02.744731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-982f2d19-4d79-45dd-94fe-94567d68cedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:02.747854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1084/2186] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:29:02 2022] DELETE /identity/v3/projects/e28426853fdb42f08d50aad76ff2e7a7 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:29:02.755099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:02.755699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:02.755825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:02.784660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:02.784660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:02.827480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:02.852916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1103/2187] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:29:02.862762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:02.863330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:02.863848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:02.895912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:02.897873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:02.911884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1085/2188] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:02.916575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:02.926992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d026-349a-4540-aee4-5770eef00f1f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:02.932975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:02.933781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:02.934885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:02.935205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:02.935978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:29:02.936317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:02.938044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:02.952003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1104/2189] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:29:02.957732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:02.967775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020456-3d7f-49ee-9591-049bbbad8ffd None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:02.973293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:02.974018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:02.974445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:02.974909 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:02.976089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:02.978507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:03.095243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1086/2190] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:29:02 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:29:03.101007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:03.113609 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524fefe-72a2-4558-9ace-3f477347f020 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:03.180926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:03.181647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:03.182016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:03.182297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:03.183024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:03.185714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:03.448617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1105/2191] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:29:03 2022] POST /identity/v3/users => generated 285 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:29:03.455537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:03.475033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185507-7e23-43a3-943b-9f8a7555d265 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:03.482611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:03.483657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:03.484171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:03.484770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:03.485886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:03.487950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:03.502439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1087/2192] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:29:03.506745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:03.518406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537d77-84c5-4fe3-91f2-970113b56109 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:03.523297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42918007-4680-497f-922a-00d76412a469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:03.524002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42918007-4680-497f-922a-00d76412a469 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:03.524315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42918007-4680-497f-922a-00d76412a469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:03.524604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42918007-4680-497f-922a-00d76412a469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:03.525352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42918007-4680-497f-922a-00d76412a469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:03.527020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42918007-4680-497f-922a-00d76412a469 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:03.539011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1106/2193] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:29:03.544873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:03.557969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be613398-e616-43e9-8797-0819470fc30c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:03.564136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:03.569736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:03.571121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:03.571783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] PATH_INFO: `/v3/projects/f975ee038b1f49b19316b222d2e74440/users/573db61c15b041559a032a1b044c6b5d/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:03.595305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f975ee038b1f49b19316b222d2e74440, user_id=573db61c15b041559a032a1b044c6b5d, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:03.598616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:03.681388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1088/2194] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:29:03 2022] PUT /identity/v3/projects/f975ee038b1f49b19316b222d2e74440/users/573db61c15b041559a032a1b044c6b5d/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:29:03.694811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8fefa-3d74-4e1a-872c-139b6d29ec97 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:03.695233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8fefa-3d74-4e1a-872c-139b6d29ec97 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:03.695641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8fefa-3d74-4e1a-872c-139b6d29ec97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:03.733809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-71b8fefa-3d74-4e1a-872c-139b6d29ec97 None None] MFA Rules not processed for user `573db61c15b041559a032a1b044c6b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:03.735520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-71b8fefa-3d74-4e1a-872c-139b6d29ec97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:03.815111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1107/2195] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:03.830596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:03.833374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:03.945348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabd47c-a38e-4364-8fe1-7862d83aa7d1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:03.949753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec51b0-78dc-4c59-8b47-7e3e0cd2e0d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:03.951949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec51b0-78dc-4c59-8b47-7e3e0cd2e0d0 service glance] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:03.951949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec51b0-78dc-4c59-8b47-7e3e0cd2e0d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:03.951949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec51b0-78dc-4c59-8b47-7e3e0cd2e0d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:03.963208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec51b0-78dc-4c59-8b47-7e3e0cd2e0d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:03.966094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec51b0-78dc-4c59-8b47-7e3e0cd2e0d0 service glance] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:03.984357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1089/2196] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:29:06.485873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bec5e-d8c4-490a-9844-6c06b8814ff8 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:06.486929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bec5e-d8c4-490a-9844-6c06b8814ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:06.486929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bec5e-d8c4-490a-9844-6c06b8814ff8 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:06.488991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1108/2197] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:29:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:29:06.498385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a68f-4426-4bc1-b7d5-9e8adbfe16c6 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:06.499494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a68f-4426-4bc1-b7d5-9e8adbfe16c6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:06.499814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a68f-4426-4bc1-b7d5-9e8adbfe16c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:06.564901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:06.576711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dc75a68f-4426-4bc1-b7d5-9e8adbfe16c6 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:06.578112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dc75a68f-4426-4bc1-b7d5-9e8adbfe16c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:06.684577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1090/2198] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:29:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:06.697243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bec5e-d8c4-490a-9844-6c06b8814ff8 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:06.710541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bec5e-d8c4-490a-9844-6c06b8814ff8 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:06.714376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ca80d-b762-450b-b5dd-ce9f9c66c2dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:06.715077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ca80d-b762-450b-b5dd-ce9f9c66c2dc service glance] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:06.715384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ca80d-b762-450b-b5dd-ce9f9c66c2dc service glance] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:06.715650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ca80d-b762-450b-b5dd-ce9f9c66c2dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:06.727484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ca80d-b762-450b-b5dd-ce9f9c66c2dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:06.729097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ca80d-b762-450b-b5dd-ce9f9c66c2dc service glance] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:06.742290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1109/2199] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:29:06.956715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387a2e-72a2-4ba6-94a3-05ea9afe3387 None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:06.956960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387a2e-72a2-4ba6-94a3-05ea9afe3387 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:06.957188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387a2e-72a2-4ba6-94a3-05ea9afe3387 None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:06.958770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1091/2200] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 04 09:29:06.967423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7f9e8-83c8-483d-9d45-58386c2d57b2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:06.967920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7f9e8-83c8-483d-9d45-58386c2d57b2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:06.968434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7f9e8-83c8-483d-9d45-58386c2d57b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:06.999511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-54f7f9e8-83c8-483d-9d45-58386c2d57b2 None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:07.001130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-54f7f9e8-83c8-483d-9d45-58386c2d57b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:07.048935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:07.107539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1110/2201] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:29:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:07.123507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387a2e-72a2-4ba6-94a3-05ea9afe3387 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:07.126007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3d387a2e-72a2-4ba6-94a3-05ea9afe3387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:07.144728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387a2e-72a2-4ba6-94a3-05ea9afe3387 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:07.149427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3804-cbc6-4e3e-a1b1-138041047bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:07.150314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3804-cbc6-4e3e-a1b1-138041047bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:07.150684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3804-cbc6-4e3e-a1b1-138041047bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:07.151049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3804-cbc6-4e3e-a1b1-138041047bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:07.170239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3804-cbc6-4e3e-a1b1-138041047bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:07.172622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3804-cbc6-4e3e-a1b1-138041047bd7 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:07.186511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1092/2202] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:29:07.321410 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf442827-994d-4f7b-a445-7db2427b78b6 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:07.322451 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf442827-994d-4f7b-a445-7db2427b78b6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:07.325655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf442827-994d-4f7b-a445-7db2427b78b6 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:07.330363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1111/2203] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:29:07 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:29:07.338125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:07.338623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:07.339102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:07.373917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:07.376762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:07.394393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1093/2204] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:29:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:07.408318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf442827-994d-4f7b-a445-7db2427b78b6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:07.425881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf442827-994d-4f7b-a445-7db2427b78b6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:07.429994 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a129a-1e08-41b3-b836-c6d11a394c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:07.430652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a129a-1e08-41b3-b836-c6d11a394c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:07.430955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a129a-1e08-41b3-b836-c6d11a394c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:07.431227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a129a-1e08-41b3-b836-c6d11a394c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:07.443697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a129a-1e08-41b3-b836-c6d11a394c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:07.445316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a129a-1e08-41b3-b836-c6d11a394c3d service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:07.458130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1112/2205] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:29:11.291678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:11.295808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:11.455051 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237377-3ee1-4977-a6c1-532ceeba0b47 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:11.465752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:11.468264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:11.468456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:11.468845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:11.472789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:11.504630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:11.506493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:11.519301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1094/2206] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:29:16.294233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd833f3-861f-46e0-a5c5-4f68853f2f6e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:16.295080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd833f3-861f-46e0-a5c5-4f68853f2f6e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:16.295080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd833f3-861f-46e0-a5c5-4f68853f2f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:16.322784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7cd833f3-861f-46e0-a5c5-4f68853f2f6e None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:16.325012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7cd833f3-861f-46e0-a5c5-4f68853f2f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:16.436152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1113/2207] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:29:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:29:16.449232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:16.451011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:16.504659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:16.543231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a63e7b-ae6e-4ac1-833b-776f3f230f36 service placement] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:16.550171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:16.551055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:16.551550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:16.551970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:16.563497 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:16.565176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:16.575064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1095/2208] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:29:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:29:26.424040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=42 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:26.591628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=16 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:42.903516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:42.904615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:42.904615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:42.925934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:42.927974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:42.941525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1114/2209] 10.222.0.46 () {58 vars in 967 bytes} [Sat Jun 4 09:29:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:42.967385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:42.978679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f9a08-67cb-4c35-b50c-857fac1a3924 service swift] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:42.984907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa1b7-3b8f-4ded-b58f-03c0086105fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:42.985572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa1b7-3b8f-4ded-b58f-03c0086105fa service swift] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:42.985925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa1b7-3b8f-4ded-b58f-03c0086105fa service swift] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:42.986207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa1b7-3b8f-4ded-b58f-03c0086105fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:42.996213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa1b7-3b8f-4ded-b58f-03c0086105fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:42.998061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa1b7-3b8f-4ded-b58f-03c0086105fa service swift] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:43.008951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1096/2210] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:29:44.113605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:44.193208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59666e9c-a5a7-4b02-acff-dfd035ebf370 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:44.199605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:44.200869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:44.201221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:44.201618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] PATH_INFO: `/v3/users/573db61c15b041559a032a1b044c6b5d` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:44.203216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573db61c15b041559a032a1b044c6b5d)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:44.205774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:44.452047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573db61c15b041559a032a1b044c6b5d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:44.465064 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573db61c15b041559a032a1b044c6b5d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:44.466236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573db61c15b041559a032a1b044c6b5d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:44.486083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573db61c15b041559a032a1b044c6b5d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:44.486949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573db61c15b041559a032a1b044c6b5d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:44.623055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1115/2211] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:29:44 2022] DELETE /identity/v3/users/573db61c15b041559a032a1b044c6b5d => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:29:44.627545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423af186-a13c-421d-8460-f103947c23c7 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:44.628230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423af186-a13c-421d-8460-f103947c23c7 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:44.628230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423af186-a13c-421d-8460-f103947c23c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:44.647745 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-423af186-a13c-421d-8460-f103947c23c7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:44.649095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-423af186-a13c-421d-8460-f103947c23c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:44.724465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1097/2212] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:44.739661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:44.742278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:44.848581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84082d77-d444-47de-97a4-18d601ee5b54 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:44.854624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45492b83-48df-4574-8965-442df2df4634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:44.855787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45492b83-48df-4574-8965-442df2df4634 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:44.856557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45492b83-48df-4574-8965-442df2df4634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:44.856969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45492b83-48df-4574-8965-442df2df4634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:44.873778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45492b83-48df-4574-8965-442df2df4634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:44.875912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45492b83-48df-4574-8965-442df2df4634 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:44.894805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1116/2213] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:29:45.404038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423af186-a13c-421d-8460-f103947c23c7 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:45.405712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-423af186-a13c-421d-8460-f103947c23c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:45.420058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423af186-a13c-421d-8460-f103947c23c7 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:45.425189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:45.426082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:45.426359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:45.426637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] PATH_INFO: `/v3/projects/f975ee038b1f49b19316b222d2e74440` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:45.427932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f975ee038b1f49b19316b222d2e74440)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:45.429973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:45.587878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f975ee038b1f49b19316b222d2e74440', 'request_id': 'req-3abdfe64-16db-4a10-bc88-7632d65136f6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:45.595049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f975ee038b1f49b19316b222d2e74440', 'request_id': 'req-3abdfe64-16db-4a10-bc88-7632d65136f6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:45.676061 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f975ee038b1f49b19316b222d2e74440', 'request_id': 'req-3abdfe64-16db-4a10-bc88-7632d65136f6'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:45.689499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] The token cache is being invalidate because project f975ee038b1f49b19316b222d2e74440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:29:45.689797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:45.690456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3abdfe64-16db-4a10-bc88-7632d65136f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:29:45.692103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1098/2214] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:29:45 2022] DELETE /identity/v3/projects/f975ee038b1f49b19316b222d2e74440 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:29:45.698080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:45.698641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:45.698641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:45.718245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:45.719494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:45.730131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:45.786032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1117/2215] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:45.795354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:45.795641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:45.795917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:45.816193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:45.818169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:45.827307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1099/2216] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:45.842450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:45.852959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed01c5-e2cf-4a38-82cc-1941cd91f2fe None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:45.858473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:45.859504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:45.860365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:45.860761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:45.861565 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:29:45.861924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:45.863619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:45.878098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1118/2217] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:29:45.883588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:45.893516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983954d8-62cf-494f-8fb0-1faa8eec9bbe None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:45.898478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:45.899170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:45.899458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:45.899737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:45.900491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:45.902444 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:46.021167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1100/2218] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:29:45 2022] POST /identity/v3/projects => generated 389 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:29:46.025670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:46.036723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee9ac6-708d-4ade-b935-f257372bcf56 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:46.100589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:46.101290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:46.102094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:46.102384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:46.103116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:46.105138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:46.196367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1119/2219] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:29:46 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:29:46.200491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:46.211944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec94ec-581f-4dfe-a414-a84298e96c7a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:46.218008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:46.229188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:46.229188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:46.229188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:46.229188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:46.229188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:46.234932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1101/2220] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:29:46.239656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:46.251288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aa1e0-5409-4dd0-a54b-69cedeae916c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:46.256536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb5192-f1d7-4bdc-ac91-9ac8b756eb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:46.256995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb5192-f1d7-4bdc-ac91-9ac8b756eb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:46.257280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb5192-f1d7-4bdc-ac91-9ac8b756eb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:46.257562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb5192-f1d7-4bdc-ac91-9ac8b756eb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:46.258307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb5192-f1d7-4bdc-ac91-9ac8b756eb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:46.259945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb5192-f1d7-4bdc-ac91-9ac8b756eb63 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:46.271880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1120/2221] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:29:46.276186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:46.294407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15799aa-ecd0-44c6-a203-5b2b29287afc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:46.300834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:46.302190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:46.302492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:46.302982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] PATH_INFO: `/v3/projects/e9ee949873db4ff5b36b1d7585558ed4/users/425df2a6bb0049d8b5d7279f13e6d648/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:46.329221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ee949873db4ff5b36b1d7585558ed4, user_id=425df2a6bb0049d8b5d7279f13e6d648, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:46.332075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:46.417811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1102/2222] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:29:46 2022] PUT /identity/v3/projects/e9ee949873db4ff5b36b1d7585558ed4/users/425df2a6bb0049d8b5d7279f13e6d648/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:29:46.433532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:46.433888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:46.434214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:46.464342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] MFA Rules not processed for user `425df2a6bb0049d8b5d7279f13e6d648`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:29:46.465573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:46.539092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1121/2223] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:29:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:29:46.550797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:29:46.552723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:29:46.564036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:46.620982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04382fb-2073-4d21-b60a-7abf4c6d3a98 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:29:46.625218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4c116-8e53-4cf5-be4f-ad0fe2a75193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:29:46.625920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4c116-8e53-4cf5-be4f-ad0fe2a75193 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:29:46.626202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4c116-8e53-4cf5-be4f-ad0fe2a75193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:29:46.626479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4c116-8e53-4cf5-be4f-ad0fe2a75193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:29:46.636584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4c116-8e53-4cf5-be4f-ad0fe2a75193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:29:46.638100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4c116-8e53-4cf5-be4f-ad0fe2a75193 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:29:46.650251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1103/2224] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:29:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:29:56.536043 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:29:56.749612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=23 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:30:07.970347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:30:08.057812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132d9f-ee9d-4fcf-ae77-012080cdc9d3 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:30:08.063295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:30:08.064250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:08.064566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:08.065137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] PATH_INFO: `/v3/users/425df2a6bb0049d8b5d7279f13e6d648` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:08.066251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425df2a6bb0049d8b5d7279f13e6d648)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:30:08.068232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:30:08.281074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425df2a6bb0049d8b5d7279f13e6d648'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:08.294573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425df2a6bb0049d8b5d7279f13e6d648'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:08.295738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425df2a6bb0049d8b5d7279f13e6d648'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:08.313981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425df2a6bb0049d8b5d7279f13e6d648'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:08.314791 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425df2a6bb0049d8b5d7279f13e6d648'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:08.429076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1122/2225] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:30:07 2022] DELETE /identity/v3/users/425df2a6bb0049d8b5d7279f13e6d648 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:30:08.433806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:08.434488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:08.434488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:08.453148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:30:08.454445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:30:08.526413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1104/2226] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:30:08.536898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:30:08.539108 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:30:08.607125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e2c9e-eab1-4fec-a502-347b2713fb49 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:30:08.611080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde32a16-be39-40ab-bb4d-948a890fa0ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:30:08.611748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32a16-be39-40ab-bb4d-948a890fa0ac service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:08.612035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32a16-be39-40ab-bb4d-948a890fa0ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:08.612349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32a16-be39-40ab-bb4d-948a890fa0ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:08.623379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde32a16-be39-40ab-bb4d-948a890fa0ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:30:08.624912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde32a16-be39-40ab-bb4d-948a890fa0ac service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:30:08.636523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1123/2227] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:30:09.205199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:30:09.206885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:30:09.220396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac879083-036d-4c03-b92b-8399da3b8b17 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:30:09.226620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:30:09.230315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:09.230728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:09.231495 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] PATH_INFO: `/v3/projects/e9ee949873db4ff5b36b1d7585558ed4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:09.234403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ee949873db4ff5b36b1d7585558ed4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:30:09.239082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:30:09.400504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ee949873db4ff5b36b1d7585558ed4', 'request_id': 'req-e27840bc-9ac4-4d91-9cc5-c85895775ef7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:09.411880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ee949873db4ff5b36b1d7585558ed4', 'request_id': 'req-e27840bc-9ac4-4d91-9cc5-c85895775ef7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:09.535942 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ee949873db4ff5b36b1d7585558ed4', 'request_id': 'req-e27840bc-9ac4-4d91-9cc5-c85895775ef7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:09.551320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] The token cache is being invalidate because project e9ee949873db4ff5b36b1d7585558ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:30:09.551827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:09.552935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e27840bc-9ac4-4d91-9cc5-c85895775ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:30:09.555530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1105/2228] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:30:09 2022] DELETE /identity/v3/projects/e9ee949873db4ff5b36b1d7585558ed4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:30:18.633529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:30:19.545391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:30:34.246909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dafe09-76f2-4eb6-bf64-15779499470a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:34.247747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dafe09-76f2-4eb6-bf64-15779499470a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:34.247920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dafe09-76f2-4eb6-bf64-15779499470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:34.279357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-01dafe09-76f2-4eb6-bf64-15779499470a None None] MFA Rules not processed for user `aeffa44fe38a4204ae21b52fe827cedc`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:30:34.281742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-01dafe09-76f2-4eb6-bf64-15779499470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:30:34.504508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1124/2229] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:30:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:30:40.035535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0682a-7866-4097-b767-7836d2483a9d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:40.035535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0682a-7866-4097-b767-7836d2483a9d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:40.035535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0682a-7866-4097-b767-7836d2483a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:40.052501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:30:40.053742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:30:40.054227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:30:40.060030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a2a0682a-7866-4097-b767-7836d2483a9d None None] MFA Rules not processed for user `34a7ef6d33b141ce8fb1cec517faffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:30:40.061857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a2a0682a-7866-4097-b767-7836d2483a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:30:40.082750 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] MFA Rules not processed for user `7b6676edfa984008b846589b2df9ef92`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:30:40.084856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:30:40.151099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1106/2230] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:30:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:30:40.188541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1125/2231] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:30:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:30:50.143384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=25 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:30:50.164817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=50 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:04.421918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:04.422726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:04.423126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:04.457880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] MFA Rules not processed for user `ac869ff528aa49989c88b3a545ba9cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:31:04.459973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:04.544550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1107/2232] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:31:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:31:13.150431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:13.152121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:13.214916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2e0d-19a5-47dd-bbe2-66631f7c43ed None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:13.218709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:13.219803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:13.220088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:13.220465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:13.222589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:13.303639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:13.304479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:13.317223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1126/2233] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:14.514442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:14.529761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828050b-599e-4efa-bfd0-4dcf78e543cc None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:14.536104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:14.537254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:14.537759 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:14.538363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:14.541559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:14.565945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:14.568740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:14.589781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1108/2234] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:23.311297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=54 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:24.580422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=31 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:30.298663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:30.300483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:30.319053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2601fa-68e6-47b7-b09c-7a03846cb9ba service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:30.325770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:30.326800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:30.327252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:30.327590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:30.329975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:30.348808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:30.351614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:30.366425 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1127/2235] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:40.359634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:40.481706 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:40.483733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:40.561752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00ae43-de78-4fbb-94a7-ca2310a49541 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:40.567409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:40.568324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:40.568710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:40.569089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:40.571357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:40.585941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:40.587420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:40.599293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1109/2236] 10.222.0.46 () {60 vars in 1396 bytes} [Sat Jun 4 09:31:40 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:42.097956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:42.099857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:42.125843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada977a-671b-48bb-bf0c-b956dc2e255e service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:42.132989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:42.133800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:42.134168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:42.134538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:42.136845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:42.150978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:42.154140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:42.172665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1128/2237] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:45.484311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:45.500830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35951877-61a4-4af4-9ead-daf0eaf98f35 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:45.506941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:45.507627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:45.507911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:45.508194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:45.510249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:45.524828 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:45.526389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:45.538840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1110/2238] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:47.361197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:47.363020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:47.375575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58e6f-4f43-490e-9913-c378cf1c9fed service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:47.379419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:47.380092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:47.380420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:47.380704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:47.382749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:47.471354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:47.473178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:47.486299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1129/2239] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:52.702835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:52.710508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:52.731645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10619f7-aa29-471a-b764-009a583e6fef service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:52.738336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:52.739673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:52.739958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:52.740655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] PATH_INFO: `/v3/users/34a7ef6d33b141ce8fb1cec517faffd1` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:52.742363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a7ef6d33b141ce8fb1cec517faffd1)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:52.745184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:52.994187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a7ef6d33b141ce8fb1cec517faffd1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:52.995158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a7ef6d33b141ce8fb1cec517faffd1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.005218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a7ef6d33b141ce8fb1cec517faffd1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.023234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a7ef6d33b141ce8fb1cec517faffd1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.024056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-887ce52e-43f5-4207-a377-0f4acdc1269d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a7ef6d33b141ce8fb1cec517faffd1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.144987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1111/2240] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:31:52 2022] DELETE /identity/v3/users/34a7ef6d33b141ce8fb1cec517faffd1 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:31:53.557988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:53.570942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec610-7610-4d34-a80a-7b06b6c8b046 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:53.671139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:53.672591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:53.673066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:53.673505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] PATH_INFO: `/v3/projects/e42983d2d3964b8cb5d32e6d653d6dfd` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:53.675919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e42983d2d3964b8cb5d32e6d653d6dfd)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:53.679638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:53.825188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e42983d2d3964b8cb5d32e6d653d6dfd', 'request_id': 'req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.833458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e42983d2d3964b8cb5d32e6d653d6dfd', 'request_id': 'req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.917385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e42983d2d3964b8cb5d32e6d653d6dfd', 'request_id': 'req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.933553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] The token cache is being invalidate because project e42983d2d3964b8cb5d32e6d653d6dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:31:53.934002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.934865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e29a044b-b1d6-4a3d-a87c-07ff31f9219c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:31:53.936910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1130/2241] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:31:53 2022] DELETE /identity/v3/projects/e42983d2d3964b8cb5d32e6d653d6dfd => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:31:53.943234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:53.944034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:53.944034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:53.978087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:31:53.980176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:54.079607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1112/2242] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:31:54.088797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.089083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.089365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.111696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:31:54.113045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:54.121883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1131/2243] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:31:54.126688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:54.131867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:54.137680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebb752-1fbe-4277-8ddb-a28d12fafae3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:54.143455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:54.144187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.144575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.144991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.145748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:31:54.146057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:54.147697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:54.166193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1113/2244] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:31:54.173279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:54.182525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:54.188700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7f8d-e2e4-4a75-a9db-8487555d1335 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:54.197031 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:54.198096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.198494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.198817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.199921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:54.203187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:54.345070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1132/2245] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:31:54 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:31:54.350413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:54.369623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41293-66d9-4d85-8772-f4fe89c05f36 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:54.455481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:54.456338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.456672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.457042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.457797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:54.460011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:54.582327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1114/2246] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:31:54 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:31:54.587007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:54.600967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41ffbc-5bae-401f-a062-7690c2275cc0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:54.606513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:54.607242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.609203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.609203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.609508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:54.611393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:54.623945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1133/2247] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:31:54.628804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:54.639962 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a14be-03b6-4106-883a-c4b1aff78656 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:54.644927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e993e-13fa-4f83-95cc-16cb56dce63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:54.645620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e993e-13fa-4f83-95cc-16cb56dce63d admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.645938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e993e-13fa-4f83-95cc-16cb56dce63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.646215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e993e-13fa-4f83-95cc-16cb56dce63d admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.646964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e993e-13fa-4f83-95cc-16cb56dce63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:54.648623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e993e-13fa-4f83-95cc-16cb56dce63d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:54.661087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1115/2248] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:31:54.665650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:54.677585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f4afd-d786-4e29-8c65-ddf1be4a8598 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:54.682463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:54.683440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.683780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.684100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] PATH_INFO: `/v3/projects/489a11d896d647b1ae00dfd37b7b717e/users/4e259ec22dd74100960d6e5f5d321e23/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.702774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489a11d896d647b1ae00dfd37b7b717e, user_id=4e259ec22dd74100960d6e5f5d321e23, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:54.705971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:54.807570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1134/2249] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:31:54 2022] PUT /identity/v3/projects/489a11d896d647b1ae00dfd37b7b717e/users/4e259ec22dd74100960d6e5f5d321e23/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:31:54.823316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:54.823668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:54.824003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:54.855784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] MFA Rules not processed for user `4e259ec22dd74100960d6e5f5d321e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:31:54.857145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:54.942756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1116/2250] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:31:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:31:54.947826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:55.019790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2258f-9e40-4abd-a594-603d945e3c66 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:55.025385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:55.026076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.026364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.026647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.027378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:55.029988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:55.166373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1135/2251] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:31:54 2022] POST /identity/v3/projects => generated 363 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:31:55.171438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:55.192078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826bb3-a84c-44a6-949d-3463b157ee68 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:55.308476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:55.317190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:55.318195 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.318658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.318983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.319935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:55.322139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:55.513741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1117/2252] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:31:55 2022] POST /identity/v3/users => generated 288 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:31:55.518941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:55.537421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9346df3-51a7-4c3e-8365-ffb590358090 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:55.546186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:55.549432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.549432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.549432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.550013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:55.553142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:55.573217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1136/2253] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:31:55.577934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:55.589658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ff164-c2fd-4e2f-9c78-0a55820bd3c2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:55.594781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:55.595717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.596001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.596278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] PATH_INFO: `/v3/projects/90af2a62b56243499b313ff68be2a200/users/6b0abf47f9ab449baca1aa88ef5eaec5/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.615154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=90af2a62b56243499b313ff68be2a200, user_id=6b0abf47f9ab449baca1aa88ef5eaec5, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:55.618038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:55.738363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1118/2254] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:31:55 2022] PUT /identity/v3/projects/90af2a62b56243499b313ff68be2a200/users/6b0abf47f9ab449baca1aa88ef5eaec5/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:31:55.742981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:55.754999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698940e5-2d61-49e8-a27f-a31bbb1553da admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:55.827619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e157e0-d065-4ff3-83c4-50294d98b106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:55.828440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e157e0-d065-4ff3-83c4-50294d98b106 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.828811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e157e0-d065-4ff3-83c4-50294d98b106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.829107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e157e0-d065-4ff3-83c4-50294d98b106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.829862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e157e0-d065-4ff3-83c4-50294d98b106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:55.832053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e157e0-d065-4ff3-83c4-50294d98b106 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:55.851621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1137/2255] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:31:55.854477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:55.866515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d22aa9-110f-48a4-acae-a34157b478ea admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:55.872018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:55.873014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.873314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.873592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] PATH_INFO: `/v3/projects/90af2a62b56243499b313ff68be2a200/users/6b0abf47f9ab449baca1aa88ef5eaec5/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.875824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90af2a62b56243499b313ff68be2a200, user_id=6b0abf47f9ab449baca1aa88ef5eaec5, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:55.878634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:55.904134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1119/2256] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:31:55 2022] PUT /identity/v3/projects/90af2a62b56243499b313ff68be2a200/users/6b0abf47f9ab449baca1aa88ef5eaec5/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:31:55.917772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:55.922857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:55.922857 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:55.958112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] MFA Rules not processed for user `6b0abf47f9ab449baca1aa88ef5eaec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:31:55.960152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:55.965206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:31:56.101145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1138/2257] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:31:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:31:56.114839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:56.116946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:31:56.202480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c07c8-0275-43c8-8ee2-9833c07bbde8 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:56.209093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:56.210253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:56.210765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:56.211293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:56.229502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:56.231297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:56.242773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1120/2258] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:56 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:31:56.503217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:31:56.521250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24e16-82c3-4b68-93a2-38a8cfb03dbc None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:31:56.528236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673611d-a77a-4797-81dd-47e243fdb0cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:31:56.530016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673611d-a77a-4797-81dd-47e243fdb0cb service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:31:56.530500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673611d-a77a-4797-81dd-47e243fdb0cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:31:56.530993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673611d-a77a-4797-81dd-47e243fdb0cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:31:56.606596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673611d-a77a-4797-81dd-47e243fdb0cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:31:56.608683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673611d-a77a-4797-81dd-47e243fdb0cb service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:31:56.618867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1139/2259] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:31:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:00.035902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:00.038597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:00.200354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8a55-e0c3-440f-bfe0-048606e9b2ed service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:00.207646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:00.208481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:00.208888 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:00.209265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:00.211602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:00.304897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:00.328472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:00.330437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:00.348503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1121/2260] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:06.614126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=54 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:07.314195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4513e-caad-4c6f-82c9-b17815f0bed2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:07.314538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4513e-caad-4c6f-82c9-b17815f0bed2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:07.314873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4513e-caad-4c6f-82c9-b17815f0bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:07.334077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-23e4513e-caad-4c6f-82c9-b17815f0bed2 None None] MFA Rules not processed for user `aeffa44fe38a4204ae21b52fe827cedc`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:07.335557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-23e4513e-caad-4c6f-82c9-b17815f0bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:07.458067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1140/2261] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:08.001958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:08.004265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:08.093417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d077b4-a374-4b08-a133-1466d07a1365 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:08.097474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:08.098203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:08.098440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:08.098986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:08.101218 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:08.113305 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:08.115204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:08.132547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1122/2262] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:08.374879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:08.375501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:08.375864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:08.403180 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:08.406068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:08.540869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1141/2263] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:32:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:08.738868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:08.742020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:08.793990 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:08.797076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:08.861527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e2adf-f1d5-420e-94f4-aec36d796be7 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:08.867959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:08.868820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:08.874149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:08.874149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:08.881320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:08.884480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:08.899797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1123/2264] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:32:08.920816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81178801-0bb3-4ee8-bc30-277145f90f20 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:08.929223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:08.930136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:08.930583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:08.931132 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:08.934445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:08.980096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:08.983337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:09.005056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1142/2265] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:32:09.575401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:09.578528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:09.604142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db826b-54bb-4bfd-950d-59b02a324cb6 service swift] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:09.612343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:09.613454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:09.613977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:09.614476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:09.635148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:09.726078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:09.729183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e207135-6d74-4ebd-b499-6b4b7d98ef1d service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:09.802614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1124/2266] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:12.176233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:12.251120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:12.299369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930d8bb-1cc7-4955-8fed-a46567eba34f service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:12.306231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:12.307138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:12.307426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:12.307759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] PATH_INFO: `/v3/users/4e259ec22dd74100960d6e5f5d321e23` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:12.312806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e259ec22dd74100960d6e5f5d321e23)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:12.315638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:12.582076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e259ec22dd74100960d6e5f5d321e23'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:12.595760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e259ec22dd74100960d6e5f5d321e23'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:12.596875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e259ec22dd74100960d6e5f5d321e23'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:12.641482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e259ec22dd74100960d6e5f5d321e23'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:12.642949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e259ec22dd74100960d6e5f5d321e23'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:12.818873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1143/2267] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:12 2022] DELETE /identity/v3/users/4e259ec22dd74100960d6e5f5d321e23 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:12.823709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:12.824657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:12.825121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:12.860595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:12.863363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:12.978571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1125/2268] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:12.992551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:12.995544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:13.005966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:13.008150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:13.029697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:13.118648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06261917-d4f6-475f-823c-e407070751d4 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:13.123472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:13.124152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:13.124466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:13.124745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:13.127914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b2edd-c7b3-4558-aa40-979dd00f5e2e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:13.132518 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:13.133208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:13.133505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:13.133957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:13.135903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:13.139882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:13.141494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:13.152829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:13.155516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:13.163916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1144/2269] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:13.165160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1126/2270] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:13.822765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:13.828251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:13.939873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b992c-c3b6-47d6-806a-1a66f02102c3 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:13.997893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:14.019280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:14.019823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:14.020342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] PATH_INFO: `/v3/projects/489a11d896d647b1ae00dfd37b7b717e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:14.037282 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489a11d896d647b1ae00dfd37b7b717e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:14.039610 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:14.286109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489a11d896d647b1ae00dfd37b7b717e', 'request_id': 'req-4901e702-9afa-4718-8a73-dbcea612ab68'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:14.312684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489a11d896d647b1ae00dfd37b7b717e', 'request_id': 'req-4901e702-9afa-4718-8a73-dbcea612ab68'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:14.323084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:14.476996 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489a11d896d647b1ae00dfd37b7b717e', 'request_id': 'req-4901e702-9afa-4718-8a73-dbcea612ab68'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:14.536992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] The token cache is being invalidate because project 489a11d896d647b1ae00dfd37b7b717e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:32:14.537466 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:14.539168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:14.542543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1145/2271] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:13 2022] DELETE /identity/v3/projects/489a11d896d647b1ae00dfd37b7b717e => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:14.547304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:14.573517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95955cd2-1351-4394-ba59-531a736b3a2d service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:14.743715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:14.746157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:14.746874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:14.747622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] PATH_INFO: `/v3/users/6b0abf47f9ab449baca1aa88ef5eaec5` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:14.750452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0abf47f9ab449baca1aa88ef5eaec5)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:14.755065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:15.048222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0abf47f9ab449baca1aa88ef5eaec5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:15.050049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0abf47f9ab449baca1aa88ef5eaec5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:15.080865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0abf47f9ab449baca1aa88ef5eaec5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:15.117224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0abf47f9ab449baca1aa88ef5eaec5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:15.117684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a51b4923-1964-46c9-ba7e-19d5c8eaee7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0abf47f9ab449baca1aa88ef5eaec5'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:15.244423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1127/2272] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:14 2022] DELETE /identity/v3/users/6b0abf47f9ab449baca1aa88ef5eaec5 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:16.086549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:16.091539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:16.137301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901e702-9afa-4718-8a73-dbcea612ab68 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:16.332878 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:16.334088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:16.334529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:16.334957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] PATH_INFO: `/v3/projects/90af2a62b56243499b313ff68be2a200` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:16.336509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90af2a62b56243499b313ff68be2a200)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:16.338849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:16.539387 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90af2a62b56243499b313ff68be2a200', 'request_id': 'req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:16.571187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90af2a62b56243499b313ff68be2a200', 'request_id': 'req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:16.677193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90af2a62b56243499b313ff68be2a200', 'request_id': 'req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:16.719072 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] The token cache is being invalidate because project 90af2a62b56243499b313ff68be2a200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:32:16.720896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:16.724067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba26d830-f31a-4a13-8ca0-cdedc3edf8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:16.734632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1146/2273] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:16 2022] DELETE /identity/v3/projects/90af2a62b56243499b313ff68be2a200 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:16.743306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:16.744214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:16.745764 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:16.842854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:16.845467 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:16.946809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:17.077770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1128/2274] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:17.078832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.079130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.079416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:17.141392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:17.144999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:17.180352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1147/2275] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:17.188357 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:17.219169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0442495-bbca-4cae-a0e2-b186fb626e9d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:17.235956 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:17.242671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.243779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.252762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:17.253781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:32:17.257712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:17.261126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:17.306346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:17.320393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1129/2276] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:32:17.331110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a8b42-eafd-469e-bf9d-3270c84df2a4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:17.339405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:17.340459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.341593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.342057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:17.343423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:17.346683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:17.512276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:17.513849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1148/2277] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:32:17 2022] POST /identity/v3/projects => generated 389 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:17.531641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da28720-5873-4911-af4c-9c1f0587dc70 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:17.675801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:17.677038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.677979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.678102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:17.679417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:17.688726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:17.810833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1130/2278] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:32:17 2022] POST /identity/v3/users => generated 295 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:17.819655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:17.845280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24e8f8-f612-4ab1-89e4-936227e18c63 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:17.855675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:17.857227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.857615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.857997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:17.860332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:17.861564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:17.879661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1149/2279] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:17.887158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:17.902426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16df5a-72fc-41a2-acd0-f28e5fdfb187 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:17.911702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19d81d-85fc-48e7-b3d3-440852129ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:17.913046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19d81d-85fc-48e7-b3d3-440852129ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.913551 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19d81d-85fc-48e7-b3d3-440852129ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.914043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19d81d-85fc-48e7-b3d3-440852129ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:17.915459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19d81d-85fc-48e7-b3d3-440852129ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:17.918545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19d81d-85fc-48e7-b3d3-440852129ed2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:17.940880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1131/2280] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:17.948315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:17.966473 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396e1f-04bf-4226-b9ee-aa3d09823b13 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:17.972982 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:17.974318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:17.974782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:17.975255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] PATH_INFO: `/v3/projects/f0cd4eeaa16b4051a74e7c5cba55526a/users/48ec71ab71d5432f8e94ec3fc2653a19/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:18.006088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cd4eeaa16b4051a74e7c5cba55526a, user_id=48ec71ab71d5432f8e94ec3fc2653a19, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:18.011665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:18.476702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1150/2281] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:17 2022] PUT /identity/v3/projects/f0cd4eeaa16b4051a74e7c5cba55526a/users/48ec71ab71d5432f8e94ec3fc2653a19/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:18.549742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:18.550252 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:18.550766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:18.599657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] MFA Rules not processed for user `48ec71ab71d5432f8e94ec3fc2653a19`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:18.601067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:18.710522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1132/2282] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:18.716799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:18.738081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:18.824225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d748-f76d-449a-9cef-30347693722e admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:18.849782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:18.850805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:18.851291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:18.851792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:18.853204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:18.861551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:19.048352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1151/2283] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:32:18 2022] POST /identity/v3/projects => generated 389 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:19.050986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:19.076344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064d09-9724-404c-8cc1-833c5b9f6110 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:19.149331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:19.217508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:19.222543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:19.222543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:19.222543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:19.222543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:19.223430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:19.386539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1133/2284] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:32:19 2022] POST /identity/v3/users => generated 301 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:32:19.393856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:19.417991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e68063-233c-42aa-a34b-7cb0cdddab7f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:19.429320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:19.434819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:19.434819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:19.434819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:19.434819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:19.436336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:19.451403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1152/2285] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:19.456362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:19.469235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac66ec-75f1-4eee-8533-3b0beaea6679 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:19.481985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:19.483037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:19.483417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:19.483766 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] PATH_INFO: `/v3/projects/04bf7a3b30d64feda10364f474be63fe/users/1759e843ebbd4bfa84251e22648ea6f4/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:19.506417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bf7a3b30d64feda10364f474be63fe, user_id=1759e843ebbd4bfa84251e22648ea6f4, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:19.526570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:19.633678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1134/2286] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:19 2022] PUT /identity/v3/projects/04bf7a3b30d64feda10364f474be63fe/users/1759e843ebbd4bfa84251e22648ea6f4/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:19.640749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:19.689652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f52a9-b02a-4325-b44a-fcccf39c026c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:19.910654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0195a5-012c-4272-a5a5-361ec403d12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:19.911850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0195a5-012c-4272-a5a5-361ec403d12b admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:19.912356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0195a5-012c-4272-a5a5-361ec403d12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:19.912832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0195a5-012c-4272-a5a5-361ec403d12b admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:19.914118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0195a5-012c-4272-a5a5-361ec403d12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:19.917268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0195a5-012c-4272-a5a5-361ec403d12b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:19.974550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:19.983209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1153/2287] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:20.003890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5efda-cca0-45a7-954c-9d24a3c35b31 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:20.020921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:20.023334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:20.024700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:20.024700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] PATH_INFO: `/v3/projects/04bf7a3b30d64feda10364f474be63fe/users/1759e843ebbd4bfa84251e22648ea6f4/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:20.028533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bf7a3b30d64feda10364f474be63fe, user_id=1759e843ebbd4bfa84251e22648ea6f4, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:20.038588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:20.066985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1135/2288] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:19 2022] PUT /identity/v3/projects/04bf7a3b30d64feda10364f474be63fe/users/1759e843ebbd4bfa84251e22648ea6f4/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:20.090061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:20.091469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:20.092216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:20.197047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] MFA Rules not processed for user `1759e843ebbd4bfa84251e22648ea6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:20.198664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:20.435787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1154/2289] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:20.437345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:20.437345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:20.499543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c523ef3-132d-41a1-925f-f7d054877e45 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:20.503655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7f415-dd08-4c3b-9ec9-ace5c03ec282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:20.504420 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f415-dd08-4c3b-9ec9-ace5c03ec282 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:20.504860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f415-dd08-4c3b-9ec9-ace5c03ec282 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:20.505170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f415-dd08-4c3b-9ec9-ace5c03ec282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:20.527531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7f415-dd08-4c3b-9ec9-ace5c03ec282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:20.529191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7f415-dd08-4c3b-9ec9-ace5c03ec282 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:20.550308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1136/2290] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:20.801394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:20.821747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:20.965311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d6ab6-2c5c-4627-90f8-31ea4baa76eb None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:20.976516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:20.977622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:20.983966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:20.983966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] PATH_INFO: `/v3/users/48ec71ab71d5432f8e94ec3fc2653a19` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:20.983966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ec71ab71d5432f8e94ec3fc2653a19)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:20.983966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:21.234492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ec71ab71d5432f8e94ec3fc2653a19'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:21.251556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ec71ab71d5432f8e94ec3fc2653a19'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:21.252476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ec71ab71d5432f8e94ec3fc2653a19'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:21.280577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ec71ab71d5432f8e94ec3fc2653a19'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:21.284199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ec71ab71d5432f8e94ec3fc2653a19'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:21.452771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1155/2291] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:20 2022] DELETE /identity/v3/users/48ec71ab71d5432f8e94ec3fc2653a19 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:21.460258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:21.461050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:21.461050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:21.481769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:21.483992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:21.562183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1137/2292] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:21.573733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:21.576930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:21.663720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f76c3-903b-4a11-8450-92d5b0490f0d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:21.670268 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:21.670959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:21.671264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:21.671546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:21.684986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:21.686728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:21.699905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1156/2293] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:22.453046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:22.455680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:22.479824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a2953-49b4-4f53-964a-d9baeb2d98a3 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:22.488820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:22.784133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:22.784133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:22.784133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] PATH_INFO: `/v3/projects/f0cd4eeaa16b4051a74e7c5cba55526a` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:22.784133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cd4eeaa16b4051a74e7c5cba55526a)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:22.784133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:22.784133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:23.100691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cd4eeaa16b4051a74e7c5cba55526a', 'request_id': 'req-6888c190-f2e9-4065-bb7c-a5151a44bcda'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.118435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cd4eeaa16b4051a74e7c5cba55526a', 'request_id': 'req-6888c190-f2e9-4065-bb7c-a5151a44bcda'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.189854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cd4eeaa16b4051a74e7c5cba55526a', 'request_id': 'req-6888c190-f2e9-4065-bb7c-a5151a44bcda'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.208953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] The token cache is being invalidate because project f0cd4eeaa16b4051a74e7c5cba55526a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:32:23.209473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.210590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.217117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1138/2294] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:22 2022] DELETE /identity/v3/projects/f0cd4eeaa16b4051a74e7c5cba55526a => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:23.223935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:23.243020 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc90b17-ac6f-400d-b087-0e8ef8c9a408 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:23.308682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:23.409660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:23.411087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:23.411557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:23.412008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] PATH_INFO: `/v3/users/1759e843ebbd4bfa84251e22648ea6f4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:23.414248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1759e843ebbd4bfa84251e22648ea6f4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:23.420792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:23.705709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1759e843ebbd4bfa84251e22648ea6f4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.734080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1759e843ebbd4bfa84251e22648ea6f4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.735872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1759e843ebbd4bfa84251e22648ea6f4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.805652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1759e843ebbd4bfa84251e22648ea6f4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:23.807097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1808074c-f739-4d7e-bd3f-6e9eb938e83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1759e843ebbd4bfa84251e22648ea6f4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:24.004631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1157/2295] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:23 2022] DELETE /identity/v3/users/1759e843ebbd4bfa84251e22648ea6f4 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:24.694852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:24.697153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:24.718661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888c190-f2e9-4065-bb7c-a5151a44bcda admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:24.816904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:24.817897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:24.818245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:24.818658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] PATH_INFO: `/v3/projects/04bf7a3b30d64feda10364f474be63fe` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:24.820561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bf7a3b30d64feda10364f474be63fe)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:24.823001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:24.958101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bf7a3b30d64feda10364f474be63fe', 'request_id': 'req-2b44bd15-2707-4134-a613-648b236b0550'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:24.966193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bf7a3b30d64feda10364f474be63fe', 'request_id': 'req-2b44bd15-2707-4134-a613-648b236b0550'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:25.079482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bf7a3b30d64feda10364f474be63fe', 'request_id': 'req-2b44bd15-2707-4134-a613-648b236b0550'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:25.093989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] The token cache is being invalidate because project 04bf7a3b30d64feda10364f474be63fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:32:25.094435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:25.095404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2b44bd15-2707-4134-a613-648b236b0550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:25.097366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1139/2296] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:24 2022] DELETE /identity/v3/projects/04bf7a3b30d64feda10364f474be63fe => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:25.103077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.103781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.103781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.124493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:25.125808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:25.200918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1158/2297] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:25.210414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.210702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.211059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.233183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:25.234514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:25.243023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1140/2298] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:25.248331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:25.259765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63715565-9178-40b0-b6d2-abcbbde40f0f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:25.265680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:25.266778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.267105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.267481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.268475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:32:25.268832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:25.270666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:25.284318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1159/2299] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:32:25.289535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:25.300659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d8bb9-418f-4d0b-8982-a82cab2e4ca0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:25.305954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:25.306647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.306928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.307208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.307951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:25.310560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:25.430042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:25.444010 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1141/2300] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:32:25 2022] POST /identity/v3/projects => generated 407 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:32:25.450365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:25.467891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112a11-09e6-453a-ab41-6a4cffd7189b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:25.509917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:25.536844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:25.537529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.537810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.538107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.538886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:25.540853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:25.683058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1160/2301] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:32:25 2022] POST /identity/v3/users => generated 304 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:32:25.688840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:25.701923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d83043-7171-4688-a058-de5fcf63bd33 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:25.707235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:25.707983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.708267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.708579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.709434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:25.711184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:25.723527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1142/2302] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:25.728874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:25.740585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10dc38-7453-4856-98e8-e2a2cbf91388 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:25.746119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037f0e2-bcad-4f71-bb21-f03a5cf19aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:25.747563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037f0e2-bcad-4f71-bb21-f03a5cf19aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.747938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037f0e2-bcad-4f71-bb21-f03a5cf19aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.748226 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037f0e2-bcad-4f71-bb21-f03a5cf19aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.749245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037f0e2-bcad-4f71-bb21-f03a5cf19aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:25.751322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037f0e2-bcad-4f71-bb21-f03a5cf19aa2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:25.766848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1161/2303] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:25.771961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:25.784382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097b2df-eb05-4b09-a8c7-a6b5b9463511 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:25.789585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:25.790651 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:25.790978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:25.791288 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] PATH_INFO: `/v3/projects/77aa8cc17494433ba7f8c7d0360dd5fb/users/9f735b1c0aec475a9e868d50ee81494a/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:25.810798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77aa8cc17494433ba7f8c7d0360dd5fb, user_id=9f735b1c0aec475a9e868d50ee81494a, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:25.813600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:26.766105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1143/2304] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:25 2022] PUT /identity/v3/projects/77aa8cc17494433ba7f8c7d0360dd5fb/users/9f735b1c0aec475a9e868d50ee81494a/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 995 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:26.783429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:26.784344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:26.784564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:26.821700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] MFA Rules not processed for user `9f735b1c0aec475a9e868d50ee81494a`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:26.823215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:26.969714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1162/2305] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:26.971237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:27.065959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd795b-29be-46b4-b189-b9a1a31db747 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:27.071485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:27.072170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.072488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.072769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.073481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:27.076152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:27.182630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1144/2306] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:32:26 2022] POST /identity/v3/projects => generated 409 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:32:27.187533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:27.200905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f8b10-90e2-43da-9be2-b599dfc3640d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:27.279383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:27.280332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.280630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.280921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.281694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:27.284104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:27.402441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1163/2307] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:32:27 2022] POST /identity/v3/users => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:27.408933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:27.427081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b287c9d-c398-4231-b8dd-7262c8954a0d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:27.433566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:27.434776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.435063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.435409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.436312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:27.438617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:27.453702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1145/2308] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:27.464876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:27.486207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defabf7-726e-4c4d-a57b-77adc8650d4d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:27.495427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:27.497794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.498311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.498836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] PATH_INFO: `/v3/projects/e5e04c5158eb430dbe3002359ea39693/users/324a8889b0844a04ac5b5899321ba1bd/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.530409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e04c5158eb430dbe3002359ea39693, user_id=324a8889b0844a04ac5b5899321ba1bd, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:27.533376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:27.648234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1164/2309] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:27 2022] PUT /identity/v3/projects/e5e04c5158eb430dbe3002359ea39693/users/324a8889b0844a04ac5b5899321ba1bd/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:27.650000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:27.671447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7dcf0-3304-4808-bf1d-a1ce8043d3a4 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:27.785827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec270ff4-39f0-4f40-9645-12aaa56fbcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:27.786912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec270ff4-39f0-4f40-9645-12aaa56fbcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.787431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec270ff4-39f0-4f40-9645-12aaa56fbcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.787959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec270ff4-39f0-4f40-9645-12aaa56fbcd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.789156 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec270ff4-39f0-4f40-9645-12aaa56fbcd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:27.792215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec270ff4-39f0-4f40-9645-12aaa56fbcd5 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:27.817304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1146/2310] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:27.819066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:27.831838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ac8ce-6a71-47eb-9237-08d3ba76644d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:27.838391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:27.840160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.841401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.841916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] PATH_INFO: `/v3/projects/e5e04c5158eb430dbe3002359ea39693/users/324a8889b0844a04ac5b5899321ba1bd/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.845997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e04c5158eb430dbe3002359ea39693, user_id=324a8889b0844a04ac5b5899321ba1bd, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:27.852762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:27.859330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:27.868533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1165/2311] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:27 2022] PUT /identity/v3/projects/e5e04c5158eb430dbe3002359ea39693/users/324a8889b0844a04ac5b5899321ba1bd/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:27.886431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:27.886813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:27.887205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:27.921783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] MFA Rules not processed for user `324a8889b0844a04ac5b5899321ba1bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:27.923643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:27.960525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:28.036435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1147/2312] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:27 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:28.056785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:28.056785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:28.163364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cbafe-1f80-4bca-aad5-04f37ec0ee54 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:28.168622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:28.169310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:28.169629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:28.169902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:28.189336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:28.191702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:28.204789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1166/2313] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:28.872814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:28.875392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:28.898380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1ba5b-4915-40b8-b763-a7649e84f268 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:28.906488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05526d0-05ef-4115-adea-bd8df481a398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:28.907631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05526d0-05ef-4115-adea-bd8df481a398 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:28.908230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05526d0-05ef-4115-adea-bd8df481a398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:28.908774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05526d0-05ef-4115-adea-bd8df481a398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:29.007362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05526d0-05ef-4115-adea-bd8df481a398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:29.008880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05526d0-05ef-4115-adea-bd8df481a398 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:29.019584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1148/2314] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:29.430993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:29.433528 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:29.635792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dae348-370f-4a10-bf4c-d867c8aa889b service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:29.646901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:29.647796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:29.648317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:29.648817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:29.651593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:29.669730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:29.671637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858a69c-e7ed-4c57-8fa3-b18b99ac570e service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:29.686237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1167/2315] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:35.963697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecc672-419b-467c-935c-e01ae46dcacb None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:35.964546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecc672-419b-467c-935c-e01ae46dcacb None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:35.964546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecc672-419b-467c-935c-e01ae46dcacb None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:35.966960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1149/2316] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:32:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:32:35.973716 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8275e1-3911-4804-a404-d3223a9701bb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:35.974385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8275e1-3911-4804-a404-d3223a9701bb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:35.974385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8275e1-3911-4804-a404-d3223a9701bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:35.996080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-be8275e1-3911-4804-a404-d3223a9701bb None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:35.997411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-be8275e1-3911-4804-a404-d3223a9701bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:36.008807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1168/2317] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:32:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:36.027012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecc672-419b-467c-935c-e01ae46dcacb None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:36.031983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-59ecc672-419b-467c-935c-e01ae46dcacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:36.110622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecc672-419b-467c-935c-e01ae46dcacb None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:36.114607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4b814-38a4-4756-89dc-3c78b77940d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:36.115278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4b814-38a4-4756-89dc-3c78b77940d7 service glance] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.115563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4b814-38a4-4756-89dc-3c78b77940d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.115842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4b814-38a4-4756-89dc-3c78b77940d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.129827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4b814-38a4-4756-89dc-3c78b77940d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:36.132770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4b814-38a4-4756-89dc-3c78b77940d7 service glance] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:36.154048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1150/2318] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:36.455045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81ab93-c211-44ba-880c-e1def1930964 None None] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.455464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81ab93-c211-44ba-880c-e1def1930964 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.455951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81ab93-c211-44ba-880c-e1def1930964 None None] PATH_INFO: `/` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.458440 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1169/2319] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:32:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:32:36.465303 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edb04f-3266-4d14-8075-3a65b5c06917 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.465689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edb04f-3266-4d14-8075-3a65b5c06917 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.466161 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edb04f-3266-4d14-8075-3a65b5c06917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.491875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-48edb04f-3266-4d14-8075-3a65b5c06917 None None] MFA Rules not processed for user `e7837f3811b04542a6e7f56e7fa118d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:36.493719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-48edb04f-3266-4d14-8075-3a65b5c06917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:36.581203 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1151/2320] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:32:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:36.597779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81ab93-c211-44ba-880c-e1def1930964 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:36.600006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0e81ab93-c211-44ba-880c-e1def1930964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:36.616842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81ab93-c211-44ba-880c-e1def1930964 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:36.622662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4132e7c-5fee-4ee1-8d2c-8de0d8830823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:36.625727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132e7c-5fee-4ee1-8d2c-8de0d8830823 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.625727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132e7c-5fee-4ee1-8d2c-8de0d8830823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.625727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132e7c-5fee-4ee1-8d2c-8de0d8830823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.638772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4132e7c-5fee-4ee1-8d2c-8de0d8830823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:36.642400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4132e7c-5fee-4ee1-8d2c-8de0d8830823 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:36.651693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1170/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:36.764730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c42904-600c-420f-a771-24c365d40b4a None None] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.765528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c42904-600c-420f-a771-24c365d40b4a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.765987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c42904-600c-420f-a771-24c365d40b4a None None] PATH_INFO: `/` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.768752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1152/2322] 10.222.0.46 () {54 vars in 921 bytes} [Sat Jun 4 09:32:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 04 09:32:36.776782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.778019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.778445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.802055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] MFA Rules not processed for user `5c1d511f4c6341e09d54bf8709efdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:36.804377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:36.820871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1171/2323] 10.222.0.46 () {58 vars in 1018 bytes} [Sat Jun 4 09:32:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:36.837023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c42904-600c-420f-a771-24c365d40b4a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:36.853168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c42904-600c-420f-a771-24c365d40b4a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:36.860461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465cf90-0f71-4666-8d96-dab8b849d857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:36.861355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465cf90-0f71-4666-8d96-dab8b849d857 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:36.861851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465cf90-0f71-4666-8d96-dab8b849d857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:36.862324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465cf90-0f71-4666-8d96-dab8b849d857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:36.885515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:36.891329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465cf90-0f71-4666-8d96-dab8b849d857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:36.894383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465cf90-0f71-4666-8d96-dab8b849d857 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:36.915788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1153/2324] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:32:40.437236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:40.518248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:40.524985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c5ea4-a3b9-48f4-a237-674f2ed3ccc4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:40.530629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:40.531971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:40.532381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:40.532764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] PATH_INFO: `/v3/users/9f735b1c0aec475a9e868d50ee81494a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:40.534442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f735b1c0aec475a9e868d50ee81494a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:40.536741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:41.039228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f735b1c0aec475a9e868d50ee81494a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:41.058026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f735b1c0aec475a9e868d50ee81494a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:41.060249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f735b1c0aec475a9e868d50ee81494a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:41.103964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f735b1c0aec475a9e868d50ee81494a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:41.105695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f735b1c0aec475a9e868d50ee81494a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:41.223809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1172/2325] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:40 2022] DELETE /identity/v3/users/9f735b1c0aec475a9e868d50ee81494a => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:32:41.229511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:41.230421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:41.230990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:41.265005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:41.267298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:41.379054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1154/2326] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:41.392447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:41.394888 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:41.529844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37f55-36cf-4c37-95c9-e02e877c415b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:41.537760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0f5e5-2dd1-4539-bf41-5b4b8a2e7b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:41.538664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0f5e5-2dd1-4539-bf41-5b4b8a2e7b2d service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:41.542289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0f5e5-2dd1-4539-bf41-5b4b8a2e7b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:41.542289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0f5e5-2dd1-4539-bf41-5b4b8a2e7b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:41.551644 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0f5e5-2dd1-4539-bf41-5b4b8a2e7b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:41.553277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0f5e5-2dd1-4539-bf41-5b4b8a2e7b2d service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:41.566828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1173/2327] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:42.251621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:42.254855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:42.279508 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79168bef-f670-4953-ae4c-ba329cbe3db4 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:42.292836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:42.295059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:42.295660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:42.296359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] PATH_INFO: `/v3/projects/77aa8cc17494433ba7f8c7d0360dd5fb` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:42.306803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77aa8cc17494433ba7f8c7d0360dd5fb)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:42.310770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:42.503076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77aa8cc17494433ba7f8c7d0360dd5fb', 'request_id': 'req-4131aa35-cff1-4361-aaca-c2edfe5b2d68'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:42.517161 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77aa8cc17494433ba7f8c7d0360dd5fb', 'request_id': 'req-4131aa35-cff1-4361-aaca-c2edfe5b2d68'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:42.545379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:42.545788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:42.546170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:42.577391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:42.579422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:42.636687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77aa8cc17494433ba7f8c7d0360dd5fb', 'request_id': 'req-4131aa35-cff1-4361-aaca-c2edfe5b2d68'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:42.675180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] The token cache is being invalidate because project 77aa8cc17494433ba7f8c7d0360dd5fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:32:42.675509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:42.676633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:42.679684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1155/2328] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:42 2022] DELETE /identity/v3/projects/77aa8cc17494433ba7f8c7d0360dd5fb => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:42.686910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:42.689649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:42.692002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1174/2329] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:32:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:42.724664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:42.728947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:42.813948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:42.892101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131aa35-cff1-4361-aaca-c2edfe5b2d68 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:42.909763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:42.911498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:42.911498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:42.911498 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] PATH_INFO: `/v3/users/324a8889b0844a04ac5b5899321ba1bd` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:42.912918 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=324a8889b0844a04ac5b5899321ba1bd)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:42.915163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:42.939219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0e1a1-5de7-4e43-b39e-ff5eedc6f943 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:42.945367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:42.948269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:42.950034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:42.950455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:42.950890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:42.963599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:42.966975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:42.980122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1175/2330] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:32:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:32:43.328496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324a8889b0844a04ac5b5899321ba1bd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:43.330736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324a8889b0844a04ac5b5899321ba1bd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:43.342417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324a8889b0844a04ac5b5899321ba1bd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:43.363866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324a8889b0844a04ac5b5899321ba1bd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:43.365297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-139ae4c8-b32c-42a7-9edb-2436461518aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324a8889b0844a04ac5b5899321ba1bd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:43.519807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1156/2331] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:42 2022] DELETE /identity/v3/users/324a8889b0844a04ac5b5899321ba1bd => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:44.518883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:44.540774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddc2e8-f3c7-46c9-9679-ea9cd5e23bf1 service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:44.745496 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:44.746594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:44.746886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:44.747179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] PATH_INFO: `/v3/projects/e5e04c5158eb430dbe3002359ea39693` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:44.748966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e04c5158eb430dbe3002359ea39693)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:44.750967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:44.972779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e04c5158eb430dbe3002359ea39693', 'request_id': 'req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:44.988201 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e04c5158eb430dbe3002359ea39693', 'request_id': 'req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:45.109278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e04c5158eb430dbe3002359ea39693', 'request_id': 'req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:45.127594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] The token cache is being invalidate because project e5e04c5158eb430dbe3002359ea39693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:32:45.127880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:45.128829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b89bab0e-1d6c-4d66-9a5b-4cc4cefa6c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:45.131169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1176/2332] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:44 2022] DELETE /identity/v3/projects/e5e04c5158eb430dbe3002359ea39693 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:45.139285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:45.139874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:45.143131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:45.170809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:45.172410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:45.316231 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1157/2333] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:45.326766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:45.327167 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:45.327392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:45.351010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:45.352383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:45.363264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1177/2334] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:45.369328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:45.393419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb14c8-4c6e-444e-8ac5-82d1448d26d1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:45.404112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:45.405002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:45.405391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:45.405796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:45.406778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:32:45.407351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:45.410025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:45.440338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1158/2335] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:32:45.445563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:45.463056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e19d21-a1e6-4b6c-b476-7c5a3d6a174d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:45.475842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:45.476646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:45.476946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:45.477345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:45.478224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:45.489443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:45.626166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1178/2336] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:32:45 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:32:45.630900 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:45.649876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4c69-daa0-40c4-a52f-74bcce038754 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:45.803058 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:45.805158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:45.805835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:45.820388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:45.820388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:45.820388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:45.973749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1159/2337] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:32:45 2022] POST /identity/v3/users => generated 286 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:45.980589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:46.000165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e924-9541-4bb3-9291-aa473e8a4c30 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:46.009063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:46.011156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:46.012187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:46.013014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:46.014520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:46.022719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:46.051283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:46.060412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1179/2338] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:46.084875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877b83a-0e26-4359-9924-c18026a5cf6b admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:46.094884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e338bc3-e125-4dd3-9e6b-e17c1c908545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:46.096064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e338bc3-e125-4dd3-9e6b-e17c1c908545 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:46.097345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e338bc3-e125-4dd3-9e6b-e17c1c908545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:46.097666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e338bc3-e125-4dd3-9e6b-e17c1c908545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:46.098984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e338bc3-e125-4dd3-9e6b-e17c1c908545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:46.102087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e338bc3-e125-4dd3-9e6b-e17c1c908545 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:46.146024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:46.147132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1160/2339] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:32:46.159325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:46.167998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fae5e-a104-4322-895f-d0ea7c3cd59a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:46.180183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:46.183257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:46.184579 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:46.186051 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] PATH_INFO: `/v3/projects/c484a76573654b54b03faf66afefcf94/users/499b1e97c8784c789a98174a26561e36/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:46.238419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c484a76573654b54b03faf66afefcf94, user_id=499b1e97c8784c789a98174a26561e36, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:46.247572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:46.745098 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1180/2340] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:46 2022] PUT /identity/v3/projects/c484a76573654b54b03faf66afefcf94/users/499b1e97c8784c789a98174a26561e36/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:32:46.753575 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e599-ac03-4622-b9a7-3ffefa1482ef None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:46.754092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e599-ac03-4622-b9a7-3ffefa1482ef None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:46.754621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e599-ac03-4622-b9a7-3ffefa1482ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:46.767769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:46.792844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-71c2e599-ac03-4622-b9a7-3ffefa1482ef None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:46.796831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-71c2e599-ac03-4622-b9a7-3ffefa1482ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:46.955093 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1161/2341] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:32:46.966060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:46.970965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:47.129168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79911b-fa64-4749-92a6-1a7401c752ff admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:47.141730 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:47.143080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:47.143558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:47.144147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:47.167725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:47.170608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:47.201741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1181/2342] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:49.732600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:49.734328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:49.735125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:49.836834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] MFA Rules not processed for user `499b1e97c8784c789a98174a26561e36`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:49.842864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:50.084054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1162/2343] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:50.089607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:50.117159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694f3c4-9d4f-4f93-a8bd-dc94744037c6 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:50.131214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:50.132324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:50.132775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:50.133127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:50.134076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:50.136869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:50.375903 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1182/2344] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:32:50 2022] POST /identity/v3/projects => generated 373 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:50.376435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:50.406881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2c72-53f0-456a-a560-f46221b1455b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:50.509558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:50.510686 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:50.511876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:50.512388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:50.513726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:50.517490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:50.749804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1163/2345] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:32:50 2022] POST /identity/v3/users => generated 293 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:32:50.756270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:50.792318 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69f33-6c7a-485b-9b90-365807629ca7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:50.803215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:50.805296 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:50.806065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:50.806592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:50.809165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:50.813384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:50.843456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1183/2346] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:50.848275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:50.873179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3412d89-b576-4b5f-8fc5-3e3869fa699b admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:50.882618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:50.884437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:50.885210 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:50.885735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] PATH_INFO: `/v3/projects/8fbc6efc8db242ca8fb1b215852b092e/users/c1bbcc4d71244509bf234906326ff422/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:50.921366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbc6efc8db242ca8fb1b215852b092e, user_id=c1bbcc4d71244509bf234906326ff422, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:50.927227 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:51.155543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:51.162928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1164/2347] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:50 2022] PUT /identity/v3/projects/8fbc6efc8db242ca8fb1b215852b092e/users/c1bbcc4d71244509bf234906326ff422/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:32:51.167479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:51.181024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d599bb9-a3c3-48b9-b52c-00efdf9f07b6 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:51.284682 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:51.285381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:51.285663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:51.285934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:51.286772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:51.300438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:51.321262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1184/2348] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:32:51.325588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:51.345160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38048f0a-ae73-42d7-b33b-859bea590c11 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:51.352390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da2c29-5dfc-433f-b5a8-2ebf42d63dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:51.353574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da2c29-5dfc-433f-b5a8-2ebf42d63dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:51.354383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da2c29-5dfc-433f-b5a8-2ebf42d63dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:51.355602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da2c29-5dfc-433f-b5a8-2ebf42d63dd9 admin admin] PATH_INFO: `/v3/projects/8fbc6efc8db242ca8fb1b215852b092e/users/c1bbcc4d71244509bf234906326ff422/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:51.360615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da2c29-5dfc-433f-b5a8-2ebf42d63dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbc6efc8db242ca8fb1b215852b092e, user_id=c1bbcc4d71244509bf234906326ff422, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:51.366919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da2c29-5dfc-433f-b5a8-2ebf42d63dd9 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:51.391354 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1165/2349] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:32:51 2022] PUT /identity/v3/projects/8fbc6efc8db242ca8fb1b215852b092e/users/c1bbcc4d71244509bf234906326ff422/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:32:51.924611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:51.928657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:52.008694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:52.016556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fefdb-a1f9-4c8c-a2d2-688d07caa6c1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:52.078026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:52.078699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:52.078980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:52.079260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:52.081431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:52.154382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:52.155987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:52.168809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1185/2350] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:32:54.513906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:54.514608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:54.514608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:54.546500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] MFA Rules not processed for user `c1bbcc4d71244509bf234906326ff422`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:54.547923 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:54.632823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1166/2351] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:32:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:32:54.651028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:54.653738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:54.728568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a635048-01f4-4ce1-aebb-c2b863de14d0 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:54.732646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:54.733314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:54.733594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:54.733879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:54.832891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:54.835829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:54.855376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1186/2352] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:32:57.284105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:57.288030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:57.315236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32104a-5c30-4baa-8429-6d199fb5e555 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:57.322881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:57.324387 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:57.325208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:57.325702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:57.350087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:57.353872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:57.484880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1167/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:58.616637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:58.625159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:58.659309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:58.833314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4f33d-e9b5-4d07-bfcb-e3492e6ce7ef service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:58.838883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:58.839563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:58.839845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:58.840125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:58.842301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:58.859211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:58.859211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:58.862564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:58.865414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:58.871478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1187/2354] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:32:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:32:58.899062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b7c3-b187-4243-adc5-c9c69a9dfa89 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:58.907656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:58.909645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:58.910722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:58.911520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] PATH_INFO: `/v3/users/ac869ff528aa49989c88b3a545ba9cf1` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:58.914499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac869ff528aa49989c88b3a545ba9cf1)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:58.920408 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:59.199157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac869ff528aa49989c88b3a545ba9cf1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:59.199878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac869ff528aa49989c88b3a545ba9cf1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:59.209207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac869ff528aa49989c88b3a545ba9cf1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:59.225617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac869ff528aa49989c88b3a545ba9cf1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:59.226715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3d8a4756-4b17-4deb-afa1-67f4ff07fcb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac869ff528aa49989c88b3a545ba9cf1'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:32:59.391469 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1168/2355] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:32:58 2022] DELETE /identity/v3/users/ac869ff528aa49989c88b3a545ba9cf1 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:32:59.706486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:32:59.721586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad068b29-51e8-4f5c-9e87-2bc4ece0fe89 service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:32:59.814692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:32:59.815889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:59.820184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:59.820184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] PATH_INFO: `/v3/projects/e27e8df485ff405e965cf22847c3b262` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:59.820184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27e8df485ff405e965cf22847c3b262)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:32:59.820762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:32:59.867358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:32:59.868370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:32:59.868865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:32:59.890615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:32:59.892130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:32:59.942565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:32:59.989379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1169/2356] 10.222.0.46 () {58 vars in 967 bytes} [Sat Jun 4 09:32:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:33:00.006627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:00.011205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:00.040103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27e8df485ff405e965cf22847c3b262', 'request_id': 'req-84bbf84f-c1eb-4d06-a786-5577b80ca932'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:00.048918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27e8df485ff405e965cf22847c3b262', 'request_id': 'req-84bbf84f-c1eb-4d06-a786-5577b80ca932'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:00.116072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41030-babc-46b4-9be0-ad74e1f711b0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:00.166909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27e8df485ff405e965cf22847c3b262', 'request_id': 'req-84bbf84f-c1eb-4d06-a786-5577b80ca932'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:00.204028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] The token cache is being invalidate because project e27e8df485ff405e965cf22847c3b262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:00.205300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:00.206694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-84bbf84f-c1eb-4d06-a786-5577b80ca932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:00.209656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1188/2357] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:32:59 2022] DELETE /identity/v3/projects/e27e8df485ff405e965cf22847c3b262 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:00.218534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503d0c0-34b5-4328-8a83-f16c57daba77 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:00.219001 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503d0c0-34b5-4328-8a83-f16c57daba77 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:00.219460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503d0c0-34b5-4328-8a83-f16c57daba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:00.273293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5503d0c0-34b5-4328-8a83-f16c57daba77 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:00.276057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5503d0c0-34b5-4328-8a83-f16c57daba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:00.322599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:00.324751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:00.324968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:00.325256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:00.331733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:00.334764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:00.441013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1189/2358] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:00.455146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:00.455748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:00.456360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:00.498342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:00.505313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:00.517196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:00.525710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1190/2359] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:33:00.526421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:00.526586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:00.538280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1170/2360] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:33:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:33:00.540778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd9908-5f3d-4f69-aeca-6bc08d959e9f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:00.550508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:00.551837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:00.552317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:00.560222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:00.560222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:33:00.560222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:00.560222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:00.600495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1191/2361] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:33:00.600787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:00.615761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe39c-21d1-4a24-af52-523e0d1c376a service swift] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:00.622886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:00.623935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:00.624470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:00.624926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:00.626089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:00.629632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:00.770683 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1171/2362] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:00 2022] POST /identity/v3/projects => generated 403 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:00.774927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:00.791674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62bbf-8227-4748-beb1-d74484324bf2 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:00.877523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:00.878343 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:00.878841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:00.879271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:00.880480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:00.883700 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:00.996230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1192/2363] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:00 2022] POST /identity/v3/users => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:01.002253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.014552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ef13-8072-4802-a908-f08da28bdd3c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.019866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.020748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.021090 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.021407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.022336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.024137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:01.037684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1172/2364] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:33:01.042855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.053806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7172e3-1fd2-47cb-b624-ae1294ce60c0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.058839 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac48993-8707-44cd-9e99-82a1216fc9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.059524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac48993-8707-44cd-9e99-82a1216fc9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.059806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac48993-8707-44cd-9e99-82a1216fc9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.060080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac48993-8707-44cd-9e99-82a1216fc9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.060883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac48993-8707-44cd-9e99-82a1216fc9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.062704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac48993-8707-44cd-9e99-82a1216fc9c1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:01.080851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1193/2365] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:01.082530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.098024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1d3ce-c608-4f99-b4a8-3a19c5451c18 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.103222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.104982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.105328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.105649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] PATH_INFO: `/v3/projects/b69a105fa33c467eac738596d855daa3/users/3a1fac61fc30483e8ee9efa115c6f4ed/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.118179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:01.126477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69a105fa33c467eac738596d855daa3, user_id=3a1fac61fc30483e8ee9efa115c6f4ed, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.131901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:01.252762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1173/2366] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:01 2022] PUT /identity/v3/projects/b69a105fa33c467eac738596d855daa3/users/3a1fac61fc30483e8ee9efa115c6f4ed/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:01.264086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.264416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.264698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.299116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] MFA Rules not processed for user `3a1fac61fc30483e8ee9efa115c6f4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:01.300811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:01.395071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1194/2367] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:01.400727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.473743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4533a-ef2a-44a6-b92d-5cda9460de98 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.478853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.480049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.480365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.480642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.481343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.483322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:01.642984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1174/2368] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:01 2022] POST /identity/v3/projects => generated 403 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:33:01.644345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.658000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3b9e0-b245-4584-8f12-984310102ad0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.677203 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:01.762745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.763970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.769340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.769340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.769340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.769340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:01.883861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1195/2369] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:01 2022] POST /identity/v3/users => generated 308 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:01.889304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.903254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebda05-a87a-4f58-9891-f23e265a2681 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.909027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.909988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.910504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.911078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.912433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.914790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:01.931297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1175/2370] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:33:01.937866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:01.950626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5300c6-6bfd-495e-ae50-d8d6675db4c5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:01.956324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:01.957445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:01.957930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:01.958493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] PATH_INFO: `/v3/projects/e12a842814e941198e7fbd488d335521/users/0c3e7e22fb83492bbcf63f6f8afd72fa/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:01.978601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12a842814e941198e7fbd488d335521, user_id=0c3e7e22fb83492bbcf63f6f8afd72fa, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:01.982149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:02.162085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1196/2371] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:01 2022] PUT /identity/v3/projects/e12a842814e941198e7fbd488d335521/users/0c3e7e22fb83492bbcf63f6f8afd72fa/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:02.167485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:02.179356 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdd64e-25dd-437f-b9cc-f312c87547ad admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:02.251270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7a3e5-f409-465e-a571-97726dbcecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:02.252528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7a3e5-f409-465e-a571-97726dbcecc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:02.253033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7a3e5-f409-465e-a571-97726dbcecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:02.253254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7a3e5-f409-465e-a571-97726dbcecc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:02.254102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7a3e5-f409-465e-a571-97726dbcecc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:02.256517 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7a3e5-f409-465e-a571-97726dbcecc1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:02.268978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1176/2372] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:02.273541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:02.286227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e8536-49e0-4eb9-bb0c-4c54008f96fb admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:02.291336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:02.292993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:02.293291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:02.293543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] PATH_INFO: `/v3/projects/e12a842814e941198e7fbd488d335521/users/0c3e7e22fb83492bbcf63f6f8afd72fa/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:02.295737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12a842814e941198e7fbd488d335521, user_id=0c3e7e22fb83492bbcf63f6f8afd72fa, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:02.298517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:02.311275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1197/2373] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:02 2022] PUT /identity/v3/projects/e12a842814e941198e7fbd488d335521/users/0c3e7e22fb83492bbcf63f6f8afd72fa/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:02.331969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:02.332255 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:02.332571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:02.364264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] MFA Rules not processed for user `0c3e7e22fb83492bbcf63f6f8afd72fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:02.365642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:02.397581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:02.473182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1177/2374] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:02.485542 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:02.487352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:02.556883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21734024-bde1-411d-9142-713ceb8c8a83 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:02.560965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddd869-4483-48a0-b661-e3ede0c96f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:02.561612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddd869-4483-48a0-b661-e3ede0c96f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:02.561889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddd869-4483-48a0-b661-e3ede0c96f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:02.562162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddd869-4483-48a0-b661-e3ede0c96f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:02.573013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddd869-4483-48a0-b661-e3ede0c96f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:02.574507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddd869-4483-48a0-b661-e3ede0c96f4e service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:02.585901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1198/2375] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:33:02.795784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:02.883330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce8d3e-1618-49a8-b71b-ed7a42792dca None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:02.893349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:02.894833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:02.895307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:02.895780 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] PATH_INFO: `/v3/users/3a1fac61fc30483e8ee9efa115c6f4ed` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:02.898261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1fac61fc30483e8ee9efa115c6f4ed)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:02.902815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:03.228213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1fac61fc30483e8ee9efa115c6f4ed'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:03.229053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1fac61fc30483e8ee9efa115c6f4ed'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:03.238158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1fac61fc30483e8ee9efa115c6f4ed'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:03.259148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1fac61fc30483e8ee9efa115c6f4ed'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:03.261174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1fac61fc30483e8ee9efa115c6f4ed'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:03.503341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1178/2376] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:02 2022] DELETE /identity/v3/users/3a1fac61fc30483e8ee9efa115c6f4ed => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:03.508323 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:03.508883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:03.508883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:03.529150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:03.530298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:03.605650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1199/2377] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:03.616263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:03.617963 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:03.669405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:03.687343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f36aef-b9fe-4395-b40e-92d30f909c57 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:03.691363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:03.692032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:03.692343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:03.692628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:03.702882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:03.704510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:03.715339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1179/2378] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:33:04.283124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:04.283690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:04.303746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8b686-b047-4360-ab5f-b36ba709e693 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:04.313143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:04.314735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:04.315316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:04.316233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] PATH_INFO: `/v3/projects/b69a105fa33c467eac738596d855daa3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:04.318926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69a105fa33c467eac738596d855daa3)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:04.323348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:04.331713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:04.487359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69a105fa33c467eac738596d855daa3', 'request_id': 'req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:04.496695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69a105fa33c467eac738596d855daa3', 'request_id': 'req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:04.650690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69a105fa33c467eac738596d855daa3', 'request_id': 'req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:04.663507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] The token cache is being invalidate because project b69a105fa33c467eac738596d855daa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:04.663807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:04.664564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:04.667019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1200/2379] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:04 2022] DELETE /identity/v3/projects/b69a105fa33c467eac738596d855daa3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:04.671635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:04.683506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b32e3-ee29-4ba8-846f-d7980ef35176 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:04.753567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:04.754610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:04.755030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:04.755337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] PATH_INFO: `/v3/users/0c3e7e22fb83492bbcf63f6f8afd72fa` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:04.756937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c3e7e22fb83492bbcf63f6f8afd72fa)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:04.760729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:05.033968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c3e7e22fb83492bbcf63f6f8afd72fa'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:05.034757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c3e7e22fb83492bbcf63f6f8afd72fa'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:05.043988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c3e7e22fb83492bbcf63f6f8afd72fa'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:05.060154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c3e7e22fb83492bbcf63f6f8afd72fa'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:05.061269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6cc3eceb-7604-4564-a64b-c675829ea13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c3e7e22fb83492bbcf63f6f8afd72fa'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:05.154867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1180/2380] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:04 2022] DELETE /identity/v3/users/0c3e7e22fb83492bbcf63f6f8afd72fa => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:05.715492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:05.718326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:05.744049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7b65-3a90-4ecf-8f53-d91dadf3745e admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:05.844763 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:05.846497 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:05.846999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:05.847526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] PATH_INFO: `/v3/projects/e12a842814e941198e7fbd488d335521` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:05.849729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12a842814e941198e7fbd488d335521)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:05.853420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:06.067092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12a842814e941198e7fbd488d335521', 'request_id': 'req-2d0b4098-c6a3-490d-a62f-0a96fee3d625'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:06.075299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12a842814e941198e7fbd488d335521', 'request_id': 'req-2d0b4098-c6a3-490d-a62f-0a96fee3d625'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:06.231737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12a842814e941198e7fbd488d335521', 'request_id': 'req-2d0b4098-c6a3-490d-a62f-0a96fee3d625'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:06.246594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] The token cache is being invalidate because project e12a842814e941198e7fbd488d335521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:06.246883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:06.247577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2d0b4098-c6a3-490d-a62f-0a96fee3d625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:06.249257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1201/2381] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:05 2022] DELETE /identity/v3/projects/e12a842814e941198e7fbd488d335521 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:06.257653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:06.258244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:06.258647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:06.283503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:06.285033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:06.353423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:06.363545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1181/2382] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:06.374541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:06.374870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:06.375169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:06.398464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:06.399798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:06.409278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1202/2383] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:06.414280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:06.425369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812fd8f-3da3-4bcf-994b-ca5ea5334928 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:06.431117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:06.432585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:06.432872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:06.433147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:06.433933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:33:06.434239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:06.435932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:06.450088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1182/2384] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:33:06.455046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:06.465385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094904d0-e7e1-4e67-87c5-bbfbee94a08a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:06.471974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:06.472809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:06.473263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:06.473688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:06.474925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:06.483187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:06.502002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:06.716350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1203/2385] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:06 2022] POST /identity/v3/projects => generated 371 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:06.722463 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:06.736352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b205338-3304-4c6d-bc4e-d3d52e02a9a5 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:06.821311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:06.822049 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:06.822368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:06.842193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:06.842193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:06.842193 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:06.970316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1183/2386] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:06 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:33:06.970538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:06.980943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0587f6-11b2-40aa-a507-336aac84c9e3 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:06.986114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:06.987006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:06.987329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:06.987661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:06.988500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:06.990197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:07.006690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1204/2387] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:07.009152 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:07.022596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5afb39-151f-403e-874c-348f91722135 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:07.028588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d1205-5bb1-40f1-8e01-1d3aaf867ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:07.029279 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d1205-5bb1-40f1-8e01-1d3aaf867ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.029564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d1205-5bb1-40f1-8e01-1d3aaf867ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.029844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d1205-5bb1-40f1-8e01-1d3aaf867ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.030592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d1205-5bb1-40f1-8e01-1d3aaf867ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:07.032385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d1205-5bb1-40f1-8e01-1d3aaf867ae9 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:07.052769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1184/2388] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:07.057673 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:07.068372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ed701-1ebb-45ae-ab99-485ef96c338d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:07.073392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:07.074293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.074575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.074863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] PATH_INFO: `/v3/projects/8a1215e6650945b3b3cf1d77bb12ee25/users/fc8fa2a1e93d4a8fb6b955654c159b2a/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.092358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1215e6650945b3b3cf1d77bb12ee25, user_id=fc8fa2a1e93d4a8fb6b955654c159b2a, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:07.095502 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:07.205023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1205/2389] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:07 2022] PUT /identity/v3/projects/8a1215e6650945b3b3cf1d77bb12ee25/users/fc8fa2a1e93d4a8fb6b955654c159b2a/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:07.219202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.219720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.220145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.266582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] MFA Rules not processed for user `fc8fa2a1e93d4a8fb6b955654c159b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:07.268674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:07.369773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1185/2390] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:07.375337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:07.465593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baffd4-218e-4f62-af7a-c5d666502dd5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:07.470840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:07.471923 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.472137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.472460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.473189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:07.475204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:07.595719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1206/2391] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:07 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:07.600436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:07.613509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03786f-7913-4a47-a74c-c312a866b4fe None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:07.674163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:07.681666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:07.682572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.682957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.683400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.684347 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:07.686552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:07.832920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1186/2392] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:07 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:07.837966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:07.860666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e30a3-0b38-4c11-9294-1f5e1408f816 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:07.868489 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:07.869987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.870300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.870670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.871591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:07.873823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:07.888774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1207/2393] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:07.894400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:07.911050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac6c31-079f-480e-8302-da563de6a689 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:07.918376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:07.923802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:07.923802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:07.923802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] PATH_INFO: `/v3/projects/478f04f30cea4a189aa9d9c5d3df376c/users/1f153619643e4636b3e8a098c207c1a2/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:07.940675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478f04f30cea4a189aa9d9c5d3df376c, user_id=1f153619643e4636b3e8a098c207c1a2, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:07.944143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:08.073454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1187/2394] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:07 2022] PUT /identity/v3/projects/478f04f30cea4a189aa9d9c5d3df376c/users/1f153619643e4636b3e8a098c207c1a2/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:08.078414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:08.091188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c553bf3-e1a8-4299-b7ed-cc5f3c2cf386 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:08.174362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffb6d-b6cd-4fa1-be4d-790a01a1a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:08.175919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffb6d-b6cd-4fa1-be4d-790a01a1a8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:08.176746 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffb6d-b6cd-4fa1-be4d-790a01a1a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:08.177423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffb6d-b6cd-4fa1-be4d-790a01a1a8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:08.179118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffb6d-b6cd-4fa1-be4d-790a01a1a8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:08.181967 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffb6d-b6cd-4fa1-be4d-790a01a1a8ee admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:08.209438 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1208/2395] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:08.214951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:08.229991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdf0a1-caab-4a11-ae58-435c2a909f37 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:08.235790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:08.236996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:08.237479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:08.237919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] PATH_INFO: `/v3/projects/478f04f30cea4a189aa9d9c5d3df376c/users/1f153619643e4636b3e8a098c207c1a2/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:08.240590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478f04f30cea4a189aa9d9c5d3df376c, user_id=1f153619643e4636b3e8a098c207c1a2, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:08.244856 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:08.256727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1188/2396] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:08 2022] PUT /identity/v3/projects/478f04f30cea4a189aa9d9c5d3df376c/users/1f153619643e4636b3e8a098c207c1a2/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:08.269329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:08.269616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:08.269795 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:08.310563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] MFA Rules not processed for user `1f153619643e4636b3e8a098c207c1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:08.312243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:08.341626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:08.413971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1209/2397] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:33:08.427897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:08.429885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:08.535840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215b4c8-20e1-4df1-8d58-72364d71a8a2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:08.541992 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:08.542832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:08.543207 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:08.543603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:08.556108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:08.559050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:08.572259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1189/2398] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:33:08.862290 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:08.875788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb27766-445f-4a8f-8343-16c034175a48 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:08.883687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:08.884041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:08.884585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:08.885607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:08.980690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:08.982332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:08.993368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1210/2399] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:18.566235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:18.988223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=48 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:19.061884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:19.063417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:19.152091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212197-f903-4cf6-a59f-85067cedc42b service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:19.157071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:19.157990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:19.158510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:19.159065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:19.161720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:19.178246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:19.179774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:19.193362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1190/2400] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:20.191951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:20.194728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:20.321075 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0036e7-00d3-4ad3-80a7-0b52f3beb867 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:20.330577 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:20.332055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:20.333514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:20.334088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] PATH_INFO: `/v3/users/499b1e97c8784c789a98174a26561e36` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:20.336528 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499b1e97c8784c789a98174a26561e36)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:20.341006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:20.599273 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499b1e97c8784c789a98174a26561e36'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:20.602861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499b1e97c8784c789a98174a26561e36'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:20.603648 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499b1e97c8784c789a98174a26561e36'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:20.620387 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499b1e97c8784c789a98174a26561e36'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:20.621628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499b1e97c8784c789a98174a26561e36'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:20.764195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1211/2401] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:20 2022] DELETE /identity/v3/users/499b1e97c8784c789a98174a26561e36 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:21.050783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:21.070349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940bc3c-5f96-422f-8b92-ba3c01be27e2 service placement] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:21.152384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:21.153528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:21.153932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:21.154240 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] PATH_INFO: `/v3/projects/c484a76573654b54b03faf66afefcf94` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:21.155621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c484a76573654b54b03faf66afefcf94)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:21.157614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:21.330457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c484a76573654b54b03faf66afefcf94', 'request_id': 'req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.339341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c484a76573654b54b03faf66afefcf94', 'request_id': 'req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.491691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c484a76573654b54b03faf66afefcf94', 'request_id': 'req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.505312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] The token cache is being invalidate because project c484a76573654b54b03faf66afefcf94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:21.506436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.507265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.510221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1191/2402] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:21 2022] DELETE /identity/v3/projects/c484a76573654b54b03faf66afefcf94 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:21.514558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:21.516200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:21.530772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1859dd-42bd-4a46-a589-eb5fb19e4740 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:21.606735 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:21.607643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:21.607930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:21.608215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] PATH_INFO: `/v3/users/c1bbcc4d71244509bf234906326ff422` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:21.609661 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1bbcc4d71244509bf234906326ff422)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:21.612154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:21.644605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:21.867044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1bbcc4d71244509bf234906326ff422'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.877297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1bbcc4d71244509bf234906326ff422'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.885011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1bbcc4d71244509bf234906326ff422'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.894815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1bbcc4d71244509bf234906326ff422'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.895825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c9dad360-5d3f-47bf-b411-b718cdde4ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1bbcc4d71244509bf234906326ff422'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:21.977217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1212/2403] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:21 2022] DELETE /identity/v3/users/c1bbcc4d71244509bf234906326ff422 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:22.356011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:22.357746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:22.373673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0e45-35d9-4c92-9213-6037ec6b5e8c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:22.397633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:22.468569 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:22.469728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:22.470154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:22.470833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] PATH_INFO: `/v3/projects/8fbc6efc8db242ca8fb1b215852b092e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:22.472623 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fbc6efc8db242ca8fb1b215852b092e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:22.474738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:22.670559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fbc6efc8db242ca8fb1b215852b092e', 'request_id': 'req-478ac9a4-18d8-4eba-85b5-89d8969d7b28'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:22.678591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fbc6efc8db242ca8fb1b215852b092e', 'request_id': 'req-478ac9a4-18d8-4eba-85b5-89d8969d7b28'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:22.876663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fbc6efc8db242ca8fb1b215852b092e', 'request_id': 'req-478ac9a4-18d8-4eba-85b5-89d8969d7b28'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:22.889330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] The token cache is being invalidate because project 8fbc6efc8db242ca8fb1b215852b092e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:22.889658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:22.890349 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-478ac9a4-18d8-4eba-85b5-89d8969d7b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:22.891974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1192/2404] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:22 2022] DELETE /identity/v3/projects/8fbc6efc8db242ca8fb1b215852b092e => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:22.897187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:22.897751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:22.897751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:22.918107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:22.919463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:22.986902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1213/2405] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:22.994233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:22.994552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:22.994840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.014588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:23.015845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:23.024701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1193/2406] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:23.036706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.047869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21640eeb-1f5e-42ce-bce8-914ffc495417 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:23.053066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:23.053742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.054026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.054304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.055215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:33:23.055533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:23.057234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:23.070753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1214/2407] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:33:23.076348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.092086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e29db7-8c34-4ada-9a4a-b47233084616 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:23.097730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:23.098493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.098774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.099232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.099972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:23.102473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:23.239310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1194/2408] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:23 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:23.243912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.254995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7827bc-a9e9-40ce-a0ea-cde185bcd6ab admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:23.323034 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:23.323742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.324022 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.324444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.325240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:23.327206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:23.560214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1215/2409] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:23 2022] POST /identity/v3/users => generated 288 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:23.566364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.578394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69321f41-3447-4685-802d-70ccd299e0c3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:23.583527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:23.584234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.584705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.584987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.585830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:23.587507 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:23.601236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1195/2410] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:23.605718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.616210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21064-cb1d-41a7-a0fe-cae638bf9bf9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:23.621594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc743fc-0185-4905-ae7a-e17140af4476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:23.622281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc743fc-0185-4905-ae7a-e17140af4476 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.622568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc743fc-0185-4905-ae7a-e17140af4476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.622848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc743fc-0185-4905-ae7a-e17140af4476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.623625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc743fc-0185-4905-ae7a-e17140af4476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:23.625321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc743fc-0185-4905-ae7a-e17140af4476 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:23.637629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1216/2411] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:23.642336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.654233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208a28-ca0a-4a4a-84cd-8c8d0169132c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:23.659292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:23.660275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.660588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.660866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] PATH_INFO: `/v3/projects/b2d96ac60cd14669b66c8a2ed0e11beb/users/726fae7dc6234f6e9ed1cc444ff9b47c/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.678118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d96ac60cd14669b66c8a2ed0e11beb, user_id=726fae7dc6234f6e9ed1cc444ff9b47c, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:23.680486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:23.782122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1196/2412] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:23 2022] PUT /identity/v3/projects/b2d96ac60cd14669b66c8a2ed0e11beb/users/726fae7dc6234f6e9ed1cc444ff9b47c/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:23.794102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:23.794620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:23.795136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:23.807408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:23.846395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] MFA Rules not processed for user `726fae7dc6234f6e9ed1cc444ff9b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:23.848594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:23.954944 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1217/2413] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:23.967068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:23.968984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:24.045455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:24.055692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecf741-28ff-47a7-acb7-37568f06458a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:24.062903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e93a61-2cf2-495b-9135-ed782fdfd78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:24.063993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93a61-2cf2-495b-9135-ed782fdfd78c service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:24.064522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93a61-2cf2-495b-9135-ed782fdfd78c service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:24.064984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93a61-2cf2-495b-9135-ed782fdfd78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:24.080813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93a61-2cf2-495b-9135-ed782fdfd78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:24.083526 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93a61-2cf2-495b-9135-ed782fdfd78c service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:24.102157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1197/2414] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:29.061087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:29.147006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41bc39-a4eb-45c1-b949-2a3527f4794f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:29.157329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:29.158946 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:29.159459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:29.160327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] PATH_INFO: `/v3/users/726fae7dc6234f6e9ed1cc444ff9b47c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:29.163181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=726fae7dc6234f6e9ed1cc444ff9b47c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:29.167576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:29.438340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726fae7dc6234f6e9ed1cc444ff9b47c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:29.446106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726fae7dc6234f6e9ed1cc444ff9b47c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:29.447450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726fae7dc6234f6e9ed1cc444ff9b47c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:29.460668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726fae7dc6234f6e9ed1cc444ff9b47c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:29.461426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726fae7dc6234f6e9ed1cc444ff9b47c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:29.576365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1218/2415] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:29 2022] DELETE /identity/v3/users/726fae7dc6234f6e9ed1cc444ff9b47c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:29.581006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:29.581829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:29.581829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:29.600056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:29.601340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:29.693012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1198/2416] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:29.705191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:29.707880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:29.727337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:29.783205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3844d0-dc69-44e7-a69d-8a8bb0ba6685 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:29.787302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e5951-0896-4748-a625-f7c64cea7311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:29.787973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e5951-0896-4748-a625-f7c64cea7311 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:29.788263 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e5951-0896-4748-a625-f7c64cea7311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:29.788592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e5951-0896-4748-a625-f7c64cea7311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:29.800986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e5951-0896-4748-a625-f7c64cea7311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:29.802561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e5951-0896-4748-a625-f7c64cea7311 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:29.814090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1219/2417] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:33:30.451084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:30.454258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:30.487031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6160c-016b-44a5-a740-a7318c763fd6 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:30.496568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:30.498617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:30.498952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:30.499819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] PATH_INFO: `/v3/projects/b2d96ac60cd14669b66c8a2ed0e11beb` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:30.502250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d96ac60cd14669b66c8a2ed0e11beb)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:30.506570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:30.659782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d96ac60cd14669b66c8a2ed0e11beb', 'request_id': 'req-c6702201-db8c-45b0-bd69-ad7daf624e03'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:30.668874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d96ac60cd14669b66c8a2ed0e11beb', 'request_id': 'req-c6702201-db8c-45b0-bd69-ad7daf624e03'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:30.752361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d96ac60cd14669b66c8a2ed0e11beb', 'request_id': 'req-c6702201-db8c-45b0-bd69-ad7daf624e03'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:30.771618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] The token cache is being invalidate because project b2d96ac60cd14669b66c8a2ed0e11beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:30.772100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:30.773263 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-c6702201-db8c-45b0-bd69-ad7daf624e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:30.776576 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1199/2418] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:30 2022] DELETE /identity/v3/projects/b2d96ac60cd14669b66c8a2ed0e11beb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:30.786742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:30.787535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:30.787535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:30.812830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:30.814401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:30.887509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1220/2419] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:30.961140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:30.961643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:30.962866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:30.995715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:30.997947 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:31.012361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1200/2420] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:31.017146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.029413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995dc23-c468-4d94-aa40-9f8567b21127 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.035044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.035776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.036057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.036384 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.037160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:33:31.037526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.039249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.054959 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1221/2421] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:33:31.060072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.070501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b175ef-a54d-4293-8b46-6b0083b584e6 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.078710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.079860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.080369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.080845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.082099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.085801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.229730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1201/2422] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:31 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:33:31.229909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.247958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ec9be-1b9c-4c70-91fc-7bcd75e888d1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.340624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.341313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.341596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.341873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.342589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.344686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.469385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1222/2423] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:31 2022] POST /identity/v3/users => generated 283 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:31.474715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.488845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32606ab6-1a6f-4071-aed3-1525c27d2a00 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.494940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.495676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.495958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.496236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.497067 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.498808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.517959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1202/2424] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:31.522006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.534800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914175e-3049-4107-b197-3eb09e1b2faf admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.539939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3be5f-b6f6-4aa8-b203-0de694b03efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.540721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3be5f-b6f6-4aa8-b203-0de694b03efb admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.541004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3be5f-b6f6-4aa8-b203-0de694b03efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.541299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3be5f-b6f6-4aa8-b203-0de694b03efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.542011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3be5f-b6f6-4aa8-b203-0de694b03efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.543654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3be5f-b6f6-4aa8-b203-0de694b03efb admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.556445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1223/2425] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:31.566603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.580065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eea9c-5731-4a41-89e8-d24e998de7c6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.591658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.594884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.594884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.594884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] PATH_INFO: `/v3/projects/7d257757a602463089cc5cb6e962ebc9/users/1101e2cdc1074373bbb8c9119974b075/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.619988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d257757a602463089cc5cb6e962ebc9, user_id=1101e2cdc1074373bbb8c9119974b075, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.622955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.708004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1203/2426] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:31 2022] PUT /identity/v3/projects/7d257757a602463089cc5cb6e962ebc9/users/1101e2cdc1074373bbb8c9119974b075/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:31.726211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.726663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.727107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.772004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] MFA Rules not processed for user `1101e2cdc1074373bbb8c9119974b075`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:31.773646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:31.787083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:31.855708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1224/2427] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:31.869323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:31.871739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:31.885209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:31.938422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7efd95-a56a-451c-aa6d-46ff66a77f20 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:31.944242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:31.946214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:31.946714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:31.947451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:31.964020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:31.966960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:31.983935 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1204/2428] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:33:33.232780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:33.235305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:33.337841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229878b3-f6bd-4a02-8c4a-1cbcfc4f81a4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:33.343083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:33.343977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:33.344448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:33.344908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:33.348161 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:33.448711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:33.450824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:33.463159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1225/2429] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:40.807129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:40.809964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:40.849598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d12d41-ea8a-4344-a247-046079b967d4 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:40.857581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:40.859201 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:40.860237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:40.860726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] PATH_INFO: `/v3/users/aeffa44fe38a4204ae21b52fe827cedc` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:40.862951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeffa44fe38a4204ae21b52fe827cedc)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:40.865633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:41.116888 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeffa44fe38a4204ae21b52fe827cedc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:41.117752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeffa44fe38a4204ae21b52fe827cedc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:41.128043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeffa44fe38a4204ae21b52fe827cedc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:41.144933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeffa44fe38a4204ae21b52fe827cedc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:41.145880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-25376ac9-39cc-4f3d-ba73-b3b7a766400c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeffa44fe38a4204ae21b52fe827cedc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:41.242050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1205/2430] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:40 2022] DELETE /identity/v3/users/aeffa44fe38a4204ae21b52fe827cedc => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:41.654824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:41.672792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7f848-6871-449b-8e64-88a76cd52cd6 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:41.766322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:41.767748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:41.768221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:41.768920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] PATH_INFO: `/v3/projects/7410684ee8db448d8385de8549bce1a7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:41.771202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7410684ee8db448d8385de8549bce1a7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:41.773876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:41.779023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:42.025455 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7410684ee8db448d8385de8549bce1a7', 'request_id': 'req-85a888d2-b080-444b-8bee-3c89886818e4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:42.034344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7410684ee8db448d8385de8549bce1a7', 'request_id': 'req-85a888d2-b080-444b-8bee-3c89886818e4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:42.134950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7410684ee8db448d8385de8549bce1a7', 'request_id': 'req-85a888d2-b080-444b-8bee-3c89886818e4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:42.149462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] The token cache is being invalidate because project 7410684ee8db448d8385de8549bce1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:42.149872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:42.150720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-85a888d2-b080-444b-8bee-3c89886818e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:42.152592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1226/2431] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:41 2022] DELETE /identity/v3/projects/7410684ee8db448d8385de8549bce1a7 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:42.160236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.160838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.160939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.193522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:42.195775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:42.293715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1206/2432] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:42.301919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921b679-39ba-435c-97aa-262d7276340c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.302428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921b679-39ba-435c-97aa-262d7276340c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.302798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921b679-39ba-435c-97aa-262d7276340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.325432 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c921b679-39ba-435c-97aa-262d7276340c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:42.326837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c921b679-39ba-435c-97aa-262d7276340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:42.335372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1227/2433] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:33:42.341399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:42.356574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd97b1-5752-4c5d-8972-46cac586d055 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:42.363516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:42.371330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.371330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.371330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.371330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:33:42.371330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:42.371330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:42.383488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1207/2434] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:33:42.389510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921b679-39ba-435c-97aa-262d7276340c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:42.400841 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921b679-39ba-435c-97aa-262d7276340c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:42.406369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:42.407057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.407469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.407788 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.408652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:42.410787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:42.519533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1228/2435] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:42 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:42.529138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:42.542265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df71d3-2ccc-4466-8b29-f53d1452d612 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:42.553510 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:42.632372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:42.633669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.634261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.634781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.636198 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:42.641264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:42.771499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1208/2436] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:42 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:42.777860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:42.792270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc21d66-2dc3-434f-a1fe-e604c99029dc admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:42.798674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:42.799698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.800106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.800582 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.801536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:42.803572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:42.819370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1229/2437] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:33:42.825000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:42.840319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ff545-17a8-4b13-8138-d80a019c24bb admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:42.848908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba6f6-eb94-499f-ad1d-db2f5dc7339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:42.850078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba6f6-eb94-499f-ad1d-db2f5dc7339a admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.850632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba6f6-eb94-499f-ad1d-db2f5dc7339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.851144 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba6f6-eb94-499f-ad1d-db2f5dc7339a admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.853186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba6f6-eb94-499f-ad1d-db2f5dc7339a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:42.855835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba6f6-eb94-499f-ad1d-db2f5dc7339a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:42.871862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1209/2438] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:33:42.876914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:42.890209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7602404-b3a3-458c-81fa-735ac602fbba admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:42.895804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:42.896757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:42.897047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:42.897327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] PATH_INFO: `/v3/projects/7b85ac9b4cc34b888bd9fb1f12b6e35b/users/f39bed050e1b4b0abff0d64735354ca0/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:42.916824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b85ac9b4cc34b888bd9fb1f12b6e35b, user_id=f39bed050e1b4b0abff0d64735354ca0, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:42.919851 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:43.056431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1230/2439] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:42 2022] PUT /identity/v3/projects/7b85ac9b4cc34b888bd9fb1f12b6e35b/users/f39bed050e1b4b0abff0d64735354ca0/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:43.069149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:43.069618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:43.070007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:43.103164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] MFA Rules not processed for user `f39bed050e1b4b0abff0d64735354ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:43.104725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:43.184654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1210/2440] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:43.196643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:43.198524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:43.295800 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50079f-f20e-475f-8349-7b059b4368d3 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:43.305118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:43.306510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:43.306940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:43.307454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:43.318067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:43.320073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:43.331614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1231/2441] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:33:51.917177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:51.919582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:52.004003 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e9176-812b-419d-b460-55c6b1445113 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:52.008964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:52.009646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:52.009985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:52.010261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:52.012366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:52.056539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:52.109553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:52.111502 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67834ece-7784-4f1c-a9e7-d2f8426ed756 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:52.124100 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1211/2442] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:52.259450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:52.278984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e7b0-7713-4a39-beef-88bb47c1f7b8 service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:52.278984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:52.281016 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:52.281349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:52.281626 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] PATH_INFO: `/v3/users/f39bed050e1b4b0abff0d64735354ca0` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:52.283118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39bed050e1b4b0abff0d64735354ca0)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:52.285193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:52.301847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:52.573619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39bed050e1b4b0abff0d64735354ca0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:52.588806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39bed050e1b4b0abff0d64735354ca0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:52.590141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39bed050e1b4b0abff0d64735354ca0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:52.621411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39bed050e1b4b0abff0d64735354ca0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:52.622914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39bed050e1b4b0abff0d64735354ca0'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:52.748989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1232/2443] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:52 2022] DELETE /identity/v3/users/f39bed050e1b4b0abff0d64735354ca0 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:52.761726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:52.762306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:52.762779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:52.795997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:52.797696 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:52.913731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1212/2444] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:33:52.926412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:52.930710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:53.036045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb643a6-c2cf-4547-b01c-f8654f02eab4 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:53.041906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3102b-305b-4e45-94bb-5085de23b43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:53.043089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3102b-305b-4e45-94bb-5085de23b43a service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:53.043602 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3102b-305b-4e45-94bb-5085de23b43a service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:53.044101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3102b-305b-4e45-94bb-5085de23b43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:53.061984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3102b-305b-4e45-94bb-5085de23b43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:53.065093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3102b-305b-4e45-94bb-5085de23b43a service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:53.088401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1233/2445] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:53.692990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:53.692990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:53.746971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dc953-3204-471f-a545-d822ce80ddec None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:53.763147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:53.765253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:53.765939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:53.766530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] PATH_INFO: `/v3/projects/7b85ac9b4cc34b888bd9fb1f12b6e35b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:53.769157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b85ac9b4cc34b888bd9fb1f12b6e35b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:53.774534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:53.975585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b85ac9b4cc34b888bd9fb1f12b6e35b', 'request_id': 'req-2bf123e8-4ed3-402f-b608-6d74615315d7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:53.982814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b85ac9b4cc34b888bd9fb1f12b6e35b', 'request_id': 'req-2bf123e8-4ed3-402f-b608-6d74615315d7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:54.180114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b85ac9b4cc34b888bd9fb1f12b6e35b', 'request_id': 'req-2bf123e8-4ed3-402f-b608-6d74615315d7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:54.195493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] The token cache is being invalidate because project 7b85ac9b4cc34b888bd9fb1f12b6e35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:54.209953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:54.209953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-2bf123e8-4ed3-402f-b608-6d74615315d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:54.209953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1213/2446] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:53 2022] DELETE /identity/v3/projects/7b85ac9b4cc34b888bd9fb1f12b6e35b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:54.212616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:54.213420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:54.213420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:54.238250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:54.239563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:54.346728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1234/2447] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:54.360204 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:54.361688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:54.361688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:54.384348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:54.385946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:54.404678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1214/2448] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:54.407065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:54.429715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f063c6-3e88-4567-8950-fa5e3b187e2d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:54.438574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:54.439674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:54.440329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:54.440769 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:54.442259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:33:54.442819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:54.447049 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:54.475289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1235/2449] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:33:54.480781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:54.499545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e26d0e-14fe-4855-acfb-51a9edae3509 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:54.510137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:54.512258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:54.513405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:54.514054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:54.515754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:54.519611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:54.672186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1215/2450] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:54 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:33:54.679186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:54.701032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b82382-442a-47c6-83c9-ffc474910e2d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:54.733718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:54.838200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:54.839284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:54.839782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:54.840264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:54.841925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:54.845807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:55.004937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1236/2451] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:54 2022] POST /identity/v3/users => generated 294 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:55.011466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:55.027044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1571485-e217-4b6e-b0ec-3e1be6109b78 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:55.034225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:55.035734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:55.046376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:55.046376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:55.046376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:55.047232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:55.075714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1216/2452] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:55.088188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:55.108441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb042c-666f-43af-ac76-d2c03798608c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:55.118904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61896c-0222-4dc3-bf32-c4242340feca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:55.126065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61896c-0222-4dc3-bf32-c4242340feca admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:55.126065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61896c-0222-4dc3-bf32-c4242340feca admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:55.126065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61896c-0222-4dc3-bf32-c4242340feca admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:55.126065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61896c-0222-4dc3-bf32-c4242340feca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:55.126756 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61896c-0222-4dc3-bf32-c4242340feca admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:55.142676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1237/2453] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:55.149524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:55.173697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec493-fe6b-4ad3-9c7f-484e6a499c20 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:55.181903 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:55.183322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:55.183608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:55.184104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] PATH_INFO: `/v3/projects/383ca2a808fd41f593ebdabce663ae67/users/bdb18e954335474bbb7ffcbc78f96429/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:55.212132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:55.215370 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383ca2a808fd41f593ebdabce663ae67, user_id=bdb18e954335474bbb7ffcbc78f96429, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:55.218290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:55.360614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1217/2454] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:55 2022] PUT /identity/v3/projects/383ca2a808fd41f593ebdabce663ae67/users/bdb18e954335474bbb7ffcbc78f96429/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:55.373393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:55.378486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:55.378486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:55.424971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] MFA Rules not processed for user `bdb18e954335474bbb7ffcbc78f96429`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:55.424971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:55.508879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1238/2455] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:55.513902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:55.613129 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c437f-ccbc-4f84-81dd-8352e430ade3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:55.618412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:55.619098 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:55.619386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:55.619667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:55.620432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:55.622637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:55.793993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1218/2456] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:33:55 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:33:55.799495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:55.827507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300dcb5-ee72-435a-9efb-2d1738d6faf6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:55.966877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:55.968480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:55.968836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:55.969581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:55.970882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:55.974981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:56.107291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1239/2457] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:33:55 2022] POST /identity/v3/users => generated 300 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:33:56.111731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:56.125037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d0a6c-6b29-4897-99aa-171df8b76844 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:56.133481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:56.133481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:56.133481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:56.133481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:56.137862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:56.137862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:56.159601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1219/2458] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:56.172842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:56.184904 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed4347-bca8-467c-a563-6172bd45661c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:56.190776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:56.191868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:56.192322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:56.192774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] PATH_INFO: `/v3/projects/d54e684a53134eba92a50e8b5eef472c/users/91d343f6eee446fbbf6abb222274f828/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:56.207359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:56.213200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54e684a53134eba92a50e8b5eef472c, user_id=91d343f6eee446fbbf6abb222274f828, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:56.216090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:56.339608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1240/2459] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:56 2022] PUT /identity/v3/projects/d54e684a53134eba92a50e8b5eef472c/users/91d343f6eee446fbbf6abb222274f828/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:33:56.345143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:56.366559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16001af4-86e3-4206-ba59-a042bff9d4d1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:56.512110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e7516-c3aa-4072-8a55-717cfdbd405a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:56.513950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7516-c3aa-4072-8a55-717cfdbd405a admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:56.514558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7516-c3aa-4072-8a55-717cfdbd405a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:56.515149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7516-c3aa-4072-8a55-717cfdbd405a admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:56.517001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e7516-c3aa-4072-8a55-717cfdbd405a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:56.520589 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e7516-c3aa-4072-8a55-717cfdbd405a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:56.550953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1220/2460] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:33:56.557845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:56.578969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a56526-23a4-446d-8ca9-674c3d37b87d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:56.591953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:56.594776 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:56.595525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:56.596091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] PATH_INFO: `/v3/projects/d54e684a53134eba92a50e8b5eef472c/users/91d343f6eee446fbbf6abb222274f828/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:56.600486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54e684a53134eba92a50e8b5eef472c, user_id=91d343f6eee446fbbf6abb222274f828, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:56.606346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:56.636350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1241/2461] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:33:56 2022] PUT /identity/v3/projects/d54e684a53134eba92a50e8b5eef472c/users/91d343f6eee446fbbf6abb222274f828/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:56.651451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:56.652228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:56.652982 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:56.709654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] MFA Rules not processed for user `91d343f6eee446fbbf6abb222274f828`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:56.711790 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:56.809352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:56.822692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1221/2462] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:56.857415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:56.860524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:56.943495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3139c2d-66c7-4237-8c72-2f4e6d439a5e admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:56.954307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311ca3d-e922-4a81-96dd-4422e9e9717f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:56.955422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311ca3d-e922-4a81-96dd-4422e9e9717f service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:56.955922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311ca3d-e922-4a81-96dd-4422e9e9717f service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:56.956589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311ca3d-e922-4a81-96dd-4422e9e9717f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:56.989281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311ca3d-e922-4a81-96dd-4422e9e9717f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:56.991886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311ca3d-e922-4a81-96dd-4422e9e9717f service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:57.013569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1242/2463] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:57.069072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:57.202500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0d4e6-b18a-4652-9459-62262636e19b None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:57.211314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:57.213212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:57.219089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:57.219089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] PATH_INFO: `/v3/users/bdb18e954335474bbb7ffcbc78f96429` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:57.219089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb18e954335474bbb7ffcbc78f96429)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:57.221537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:57.620172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb18e954335474bbb7ffcbc78f96429'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:57.621403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb18e954335474bbb7ffcbc78f96429'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:57.644811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb18e954335474bbb7ffcbc78f96429'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:57.682343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb18e954335474bbb7ffcbc78f96429'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:57.683384 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb18e954335474bbb7ffcbc78f96429'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:57.786908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1222/2464] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:57 2022] DELETE /identity/v3/users/bdb18e954335474bbb7ffcbc78f96429 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:57.793690 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:57.799158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:57.799158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:57.826364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:57.828599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:57.943345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1243/2465] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:33:57.967996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:57.967996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:58.075856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:58.078752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:58.110586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed560c9-ff9e-4449-aa29-28f36f0fe64e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:58.116624 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:58.118460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:58.118668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148627e-ad31-4b52-a3ab-d2fbc7db7588 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:58.118933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:58.119317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:58.136041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:58.137854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:58.138596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:58.139184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] PATH_INFO: `/v3/users/7b6676edfa984008b846589b2df9ef92` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:58.141544 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6676edfa984008b846589b2df9ef92)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:58.143618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:58.145221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:58.145422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:58.159134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1223/2466] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:33:58.477227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6676edfa984008b846589b2df9ef92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:58.489685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6676edfa984008b846589b2df9ef92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:58.490510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6676edfa984008b846589b2df9ef92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:58.506378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6676edfa984008b846589b2df9ef92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:58.507143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6676edfa984008b846589b2df9ef92'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:58.509687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:58.637919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1244/2467] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:58 2022] DELETE /identity/v3/users/7b6676edfa984008b846589b2df9ef92 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:58.855291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:58.857804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:58.905643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4c27b-3735-49d4-abe7-ca5e72c2fb6f service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:58.913085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:33:58.981215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:58.982095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:58.982429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:58.982754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] PATH_INFO: `/v3/projects/383ca2a808fd41f593ebdabce663ae67` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:58.984212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383ca2a808fd41f593ebdabce663ae67)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:58.986556 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:59.311221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:59.315065 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:59.342036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383ca2a808fd41f593ebdabce663ae67', 'request_id': 'req-f2817d35-6bb2-4e6c-abc3-92db3b819903'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.354729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e626844-79e6-46a4-9679-14f60294a39d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:59.357376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383ca2a808fd41f593ebdabce663ae67', 'request_id': 'req-f2817d35-6bb2-4e6c-abc3-92db3b819903'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.479340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:59.480833 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:59.481295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:59.481957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] PATH_INFO: `/v3/projects/0cf07261800e47938626f61a1fb73b2b` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:59.484243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf07261800e47938626f61a1fb73b2b)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:59.487664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:59.523657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383ca2a808fd41f593ebdabce663ae67', 'request_id': 'req-f2817d35-6bb2-4e6c-abc3-92db3b819903'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.546921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] The token cache is being invalidate because project 383ca2a808fd41f593ebdabce663ae67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:59.547381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.553832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.556613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1224/2468] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:58 2022] DELETE /identity/v3/projects/383ca2a808fd41f593ebdabce663ae67 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:59.565341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:33:59.567142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:59.590237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817d35-6bb2-4e6c-abc3-92db3b819903 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:33:59.607873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:33:59.607873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:59.607873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:59.607873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] PATH_INFO: `/v3/users/91d343f6eee446fbbf6abb222274f828` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:59.608535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d343f6eee446fbbf6abb222274f828)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:33:59.608535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:33:59.714635 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf07261800e47938626f61a1fb73b2b', 'request_id': 'req-dc982c5c-7d00-4ffd-a199-fd0e13da870f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.722909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf07261800e47938626f61a1fb73b2b', 'request_id': 'req-dc982c5c-7d00-4ffd-a199-fd0e13da870f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.869632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf07261800e47938626f61a1fb73b2b', 'request_id': 'req-dc982c5c-7d00-4ffd-a199-fd0e13da870f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.884086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] The token cache is being invalidate because project 0cf07261800e47938626f61a1fb73b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:33:59.884437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.885285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-dc982c5c-7d00-4ffd-a199-fd0e13da870f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.886899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1245/2469] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:33:59 2022] DELETE /identity/v3/projects/0cf07261800e47938626f61a1fb73b2b => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:33:59.892581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c0971-04de-49a7-affb-55d6d4c160cb None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:33:59.892926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c0971-04de-49a7-affb-55d6d4c160cb None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:33:59.893257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c0971-04de-49a7-affb-55d6d4c160cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:33:59.922086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-c12c0971-04de-49a7-affb-55d6d4c160cb None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:33:59.924090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c12c0971-04de-49a7-affb-55d6d4c160cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:33:59.928710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d343f6eee446fbbf6abb222274f828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.930024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d343f6eee446fbbf6abb222274f828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.941030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d343f6eee446fbbf6abb222274f828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.960669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d343f6eee446fbbf6abb222274f828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:33:59.962554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d343f6eee446fbbf6abb222274f828'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:00.006257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1246/2470] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:00.020193 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.020555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.020835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.041638 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:00.042913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:00.053105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1247/2471] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:00.058468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.073270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aba4f-f458-400b-a2ae-61cad968f9f7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.076145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.077158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.077500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.077850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.078825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:00.079172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.080917 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.096118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1248/2472] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:34:00.102298 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.105866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1225/2473] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:33:59 2022] DELETE /identity/v3/users/91d343f6eee446fbbf6abb222274f828 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:00.114517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:00.120358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5172f76-367b-4d4e-9ced-54faebc598f7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.214269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.215136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.215564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.215986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.216968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.220179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.332066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1249/2474] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:00 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:00.336991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.349355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21774d-2f86-40a9-8f7b-6adf4b4af7e7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.355753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:00.440424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.441636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.442145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.442769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.444355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.448979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.594060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1226/2475] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:00 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:00.596919 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.613636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85258a-9392-4a76-b422-63698b115766 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.621179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.622313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.623110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.623882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.625118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.627969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.647278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1250/2476] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:00.652785 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.671871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff0815-f4f6-413c-893e-4a1636430100 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.680644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.682068 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.682978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.683893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.685313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.688617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.715394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1227/2477] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:00.719721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.741204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e800d7e-773a-4d26-a637-9c75169233ec admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.765807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e36d08-27c2-43c4-a691-bc7372e05ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.767829 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e36d08-27c2-43c4-a691-bc7372e05ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.768732 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e36d08-27c2-43c4-a691-bc7372e05ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.770856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e36d08-27c2-43c4-a691-bc7372e05ad7 admin admin] PATH_INFO: `/v3/projects/d5d890325797406ba7a5bdb263c41713/users/b72b94a289cf462c8a9251e1d2866288/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.792132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:00.815704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:00.816693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e36d08-27c2-43c4-a691-bc7372e05ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d890325797406ba7a5bdb263c41713, user_id=b72b94a289cf462c8a9251e1d2866288, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.822976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e36d08-27c2-43c4-a691-bc7372e05ad7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.861901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15435da8-93e0-4c2d-b986-618c0c145a51 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:00.867993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:00.869394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:00.869843 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:00.870392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] PATH_INFO: `/v3/projects/d54e684a53134eba92a50e8b5eef472c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:00.874407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54e684a53134eba92a50e8b5eef472c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:00.891364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:00.988133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1251/2478] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:00 2022] PUT /identity/v3/projects/d5d890325797406ba7a5bdb263c41713/users/b72b94a289cf462c8a9251e1d2866288/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:01.003027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.003462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.003871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.053858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] MFA Rules not processed for user `b72b94a289cf462c8a9251e1d2866288`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:01.055956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:01.166002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54e684a53134eba92a50e8b5eef472c', 'request_id': 'req-7678c013-9910-45d9-9cb3-7c68ddc0eb90'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:01.186815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54e684a53134eba92a50e8b5eef472c', 'request_id': 'req-7678c013-9910-45d9-9cb3-7c68ddc0eb90'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:01.191159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1252/2479] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:01.207815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:01.210459 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:01.285029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54e684a53134eba92a50e8b5eef472c', 'request_id': 'req-7678c013-9910-45d9-9cb3-7c68ddc0eb90'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:01.316585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] The token cache is being invalidate because project d54e684a53134eba92a50e8b5eef472c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:01.317071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:01.318084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7678c013-9910-45d9-9cb3-7c68ddc0eb90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:01.321442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1228/2480] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:00 2022] DELETE /identity/v3/projects/d54e684a53134eba92a50e8b5eef472c => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:01.327661 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc20e-79de-4469-a582-920ec56c4d69 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.327964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc20e-79de-4469-a582-920ec56c4d69 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.328461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc20e-79de-4469-a582-920ec56c4d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.329177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:01.359224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-632bc20e-79de-4469-a582-920ec56c4d69 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:01.360958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-632bc20e-79de-4469-a582-920ec56c4d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:01.362057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bbb7f-d63c-4e2f-a63c-cd4de15749b9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:01.446404 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:01.447538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.448081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.448546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.451104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:01.470645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1229/2481] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:01.483556 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.484040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.484548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.517237 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:01.519271 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:01.534619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1230/2482] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:01.542565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:01.551486 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:01.556146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b251e7-6f2c-4956-a9a2-89a7a23a9c86 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:01.562637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:01.566702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:01.569810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:01.569810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.569810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.569810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.571169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:01.571685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:01.573830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:01.593235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1231/2483] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:34:01.596278 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1253/2484] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:01.599855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:01.615727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17dd2-1ef2-4df8-b635-779fcfe468c9 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:01.621581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:01.622572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.623107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.623774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.625279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:01.628961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:01.793940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1254/2485] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:01 2022] POST /identity/v3/projects => generated 399 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:01.799009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:01.810350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6ae9-e395-4c8d-ba2c-d7ed21fb70cb admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:01.887596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:01.888307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:01.888611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:01.888890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:01.889599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:01.891693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:02.044906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1232/2486] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:01 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:02.049484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:02.064866 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edb46d-83af-4f4c-b084-1abc865a8769 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:02.073486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:02.074313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:02.074709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:02.075101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:02.076024 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:02.078623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:02.094695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1255/2487] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:02.097753 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:02.110361 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f837-bdcb-4e7b-977f-8a647639f80e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:02.116488 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76081342-45c9-4261-a9f1-4201b2098543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:02.117748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76081342-45c9-4261-a9f1-4201b2098543 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:02.118034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76081342-45c9-4261-a9f1-4201b2098543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:02.118310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76081342-45c9-4261-a9f1-4201b2098543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:02.119072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76081342-45c9-4261-a9f1-4201b2098543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:02.120787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76081342-45c9-4261-a9f1-4201b2098543 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:02.132202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1233/2488] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:02.137260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:02.154913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d45a-3756-4537-b9a2-d1ceaa1760e5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:02.163039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:02.166015 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:02.166836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:02.167347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] PATH_INFO: `/v3/projects/4d85c9e05c4d469a8113bd9bd76e158a/users/abc0912c2e6b48cfaaf39c2cee5287e9/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:02.196274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d85c9e05c4d469a8113bd9bd76e158a, user_id=abc0912c2e6b48cfaaf39c2cee5287e9, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:02.201848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:02.336070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1256/2489] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:02 2022] PUT /identity/v3/projects/4d85c9e05c4d469a8113bd9bd76e158a/users/abc0912c2e6b48cfaaf39c2cee5287e9/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:02.347899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:02.348601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:02.348601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:02.384521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] MFA Rules not processed for user `abc0912c2e6b48cfaaf39c2cee5287e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:02.386291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:02.506182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1234/2490] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:02.512627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:02.593852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:02.599955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8d5e-3ecc-4f3c-a4d0-1fe344d03345 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:02.607752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:02.609125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:02.609629 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:02.610169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:02.612008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:02.616329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:02.773830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1257/2491] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:02 2022] POST /identity/v3/projects => generated 399 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:02.778709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:02.806175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d0bb1-e647-4a4b-92a9-bcd33a2a4f5e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:02.889870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:02.890940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:02.891436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:02.891925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:02.893300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:02.897331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:02.908472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:02.998136 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1235/2492] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:02 2022] POST /identity/v3/users => generated 306 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:03.002892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:03.015377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00324de0-0e2c-4d06-a23c-7c4fc62f9933 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:03.020843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:03.021526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.021813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.022107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:03.022886 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:03.025209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:03.037514 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1258/2493] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:03.042939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:03.062268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a37046-97e6-4443-bcb6-39d1d7e37761 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:03.070972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:03.072625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.073250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.073927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] PATH_INFO: `/v3/projects/56b557aa64f24511a33a15227e8279d8/users/029047e026054860bd286fcc68a1ff38/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:03.095604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b557aa64f24511a33a15227e8279d8, user_id=029047e026054860bd286fcc68a1ff38, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:03.098474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:03.192602 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1236/2494] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:03 2022] PUT /identity/v3/projects/56b557aa64f24511a33a15227e8279d8/users/029047e026054860bd286fcc68a1ff38/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:03.197107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:03.208380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e29d-04e0-4cf3-8350-35279e539463 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:03.305481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc177f6-85f2-46cd-8102-0ace69f623c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:03.306659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc177f6-85f2-46cd-8102-0ace69f623c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.307172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc177f6-85f2-46cd-8102-0ace69f623c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.307657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc177f6-85f2-46cd-8102-0ace69f623c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:03.309089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc177f6-85f2-46cd-8102-0ace69f623c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:03.312189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc177f6-85f2-46cd-8102-0ace69f623c8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:03.331985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1259/2495] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:03.338180 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:03.357440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f648b8-eb03-45a4-858c-ca47dc22206a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:03.365550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:03.367164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.367734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.368215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] PATH_INFO: `/v3/projects/56b557aa64f24511a33a15227e8279d8/users/029047e026054860bd286fcc68a1ff38/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:03.371941 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b557aa64f24511a33a15227e8279d8, user_id=029047e026054860bd286fcc68a1ff38, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:03.377326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:03.394232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1237/2496] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:03 2022] PUT /identity/v3/projects/56b557aa64f24511a33a15227e8279d8/users/029047e026054860bd286fcc68a1ff38/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:03.406670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.406951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.407229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:03.438191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] MFA Rules not processed for user `029047e026054860bd286fcc68a1ff38`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:03.439505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:03.522289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1260/2497] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:03 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:03.539335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:03.541878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:03.626619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874558f-2269-43d5-8d69-b8c0667b8692 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:03.630957 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:03.631629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.631915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.632194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:03.644007 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:03.645794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:03.657939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1238/2498] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:03.946006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:03.956908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a765b8-8c92-4ffa-865a-716bf3cf25b5 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:03.961076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:03.961803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:03.962149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:03.962471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:04.020899 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:04.032710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:04.034414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:04.045653 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1261/2499] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:10.197818 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:10.199460 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:10.334291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0fa33-4979-4c8a-8614-ef5505ab14cb service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:10.345340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:10.347429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:10.348089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:10.348583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] PATH_INFO: `/v3/users/fc8fa2a1e93d4a8fb6b955654c159b2a` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:10.350728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc8fa2a1e93d4a8fb6b955654c159b2a)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:10.359674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:10.474135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:10.500784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b690448-4760-40db-9d93-11839bafba2e service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:10.509289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:10.510813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:10.511254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:10.511740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] PATH_INFO: `/v3/users/abc0912c2e6b48cfaaf39c2cee5287e9` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:10.514177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc0912c2e6b48cfaaf39c2cee5287e9)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:10.518228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:10.605440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc8fa2a1e93d4a8fb6b955654c159b2a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.605933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc8fa2a1e93d4a8fb6b955654c159b2a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.614352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc8fa2a1e93d4a8fb6b955654c159b2a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.615770 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:10.629452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc8fa2a1e93d4a8fb6b955654c159b2a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.630256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-75bc837c-f0cf-42e7-a10b-76b87eaac9fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc8fa2a1e93d4a8fb6b955654c159b2a'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.733581 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1239/2500] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:10 2022] DELETE /identity/v3/users/fc8fa2a1e93d4a8fb6b955654c159b2a => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:10.740853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:10.741409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:10.742590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:10.767650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:10.769017 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:10.773493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc0912c2e6b48cfaaf39c2cee5287e9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.788519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc0912c2e6b48cfaaf39c2cee5287e9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.790409 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc0912c2e6b48cfaaf39c2cee5287e9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.812378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc0912c2e6b48cfaaf39c2cee5287e9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.813289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b29c9681-750b-487d-92d3-16bf73ce5106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc0912c2e6b48cfaaf39c2cee5287e9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:10.872246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1240/2501] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:10.883684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:10.885648 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:10.907157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1262/2502] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:10 2022] DELETE /identity/v3/users/abc0912c2e6b48cfaaf39c2cee5287e9 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:10.908775 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:10.909091 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:10.909375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:10.937559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:10.939618 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:10.983219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34320e-dd04-44ef-989b-4acc431b2a08 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:11.055840 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1263/2503] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:11.058773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:11.059659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:11.060078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:11.060558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:11.062663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:11.072461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:11.073848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e9a0b-e28e-4006-ac7c-2a8c0cb88a02 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:11.074245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:11.077856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:11.078520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:11.078806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:11.079094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:11.086561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1241/2504] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:11.089819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:11.091300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:11.105158 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1264/2505] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:11.783400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:11.785825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:11.828798 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:11.838308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:11.839832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:11.861004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e017d5a-e513-4be9-ad9b-969476b14954 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:11.866548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:11.868549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:11.869131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:11.869731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] PATH_INFO: `/v3/projects/4d85c9e05c4d469a8113bd9bd76e158a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:11.871325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d85c9e05c4d469a8113bd9bd76e158a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:11.873510 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:11.921024 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6134cb-a041-461d-b388-8640ad8016cc service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:11.927767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:11.929403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:11.929689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:11.929966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:11.932084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:11.949812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:11.952914 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:11.976540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1242/2506] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:11.978388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:11.981134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:12.007722 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a831e7-e5a7-4e86-b838-dd31fdb4e8e3 service placement] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:12.015416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:12.031077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d85c9e05c4d469a8113bd9bd76e158a', 'request_id': 'req-167448b4-e941-4dbf-be69-0ec1c85b2406'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.043598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d85c9e05c4d469a8113bd9bd76e158a', 'request_id': 'req-167448b4-e941-4dbf-be69-0ec1c85b2406'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.125836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:12.127313 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:12.127775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:12.128274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] PATH_INFO: `/v3/projects/8a1215e6650945b3b3cf1d77bb12ee25` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:12.130462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1215e6650945b3b3cf1d77bb12ee25)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:12.133454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:12.157623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d85c9e05c4d469a8113bd9bd76e158a', 'request_id': 'req-167448b4-e941-4dbf-be69-0ec1c85b2406'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.170045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] The token cache is being invalidate because project 4d85c9e05c4d469a8113bd9bd76e158a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:12.170335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.171027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.172864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1265/2507] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:11 2022] DELETE /identity/v3/projects/4d85c9e05c4d469a8113bd9bd76e158a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:12.177077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:12.178712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:12.194321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167448b4-e941-4dbf-be69-0ec1c85b2406 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:12.201041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:12.202339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:12.202852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:12.203285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] PATH_INFO: `/v3/users/029047e026054860bd286fcc68a1ff38` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:12.208684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=029047e026054860bd286fcc68a1ff38)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:12.211332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:12.336431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1215e6650945b3b3cf1d77bb12ee25', 'request_id': 'req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.345713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1215e6650945b3b3cf1d77bb12ee25', 'request_id': 'req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.517260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1215e6650945b3b3cf1d77bb12ee25', 'request_id': 'req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.537747 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] The token cache is being invalidate because project 8a1215e6650945b3b3cf1d77bb12ee25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:12.538345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.539223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.541503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1243/2508] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:11 2022] DELETE /identity/v3/projects/8a1215e6650945b3b3cf1d77bb12ee25 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:12.547796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:12.550038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:12.641779 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029047e026054860bd286fcc68a1ff38'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.644269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:12.652561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029047e026054860bd286fcc68a1ff38'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.653524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029047e026054860bd286fcc68a1ff38'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.666447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1e2e3-a5fd-4ad7-bbfd-d785e5dc0367 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:12.676305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029047e026054860bd286fcc68a1ff38'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.677504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029047e026054860bd286fcc68a1ff38'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.680841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:12.682519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:12.683026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:12.687901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] PATH_INFO: `/v3/users/1f153619643e4636b3e8a098c207c1a2` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:12.687901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f153619643e4636b3e8a098c207c1a2)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:12.690748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:12.776267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1266/2509] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:12 2022] DELETE /identity/v3/users/029047e026054860bd286fcc68a1ff38 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:12.974744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f153619643e4636b3e8a098c207c1a2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.975470 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f153619643e4636b3e8a098c207c1a2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:12.985479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f153619643e4636b3e8a098c207c1a2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.002183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f153619643e4636b3e8a098c207c1a2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.002685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f153619643e4636b3e8a098c207c1a2'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.106743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1244/2510] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:12 2022] DELETE /identity/v3/users/1f153619643e4636b3e8a098c207c1a2 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:13.278276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:13.280861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:13.405142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071e37-0e7d-402e-808b-9c5801d4f4b3 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:13.485667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:13.499095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:13.500260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:13.500754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:13.501253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] PATH_INFO: `/v3/projects/56b557aa64f24511a33a15227e8279d8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:13.502980 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b557aa64f24511a33a15227e8279d8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:13.505600 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:13.657407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b557aa64f24511a33a15227e8279d8', 'request_id': 'req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.666027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b557aa64f24511a33a15227e8279d8', 'request_id': 'req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.686429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:13.688110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:13.706751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de7e09-304c-4ff0-9cd5-cf93f317ecfc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:13.762213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b557aa64f24511a33a15227e8279d8', 'request_id': 'req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.783441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] The token cache is being invalidate because project 56b557aa64f24511a33a15227e8279d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:13.784013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.785184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589a9cb5-3314-4b05-a5f2-01a9e36c4cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.787936 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1267/2511] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:13 2022] DELETE /identity/v3/projects/56b557aa64f24511a33a15227e8279d8 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:13.796088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:13.796454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe288-53cf-44da-a032-c364bb40d21f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:13.796688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe288-53cf-44da-a032-c364bb40d21f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:13.797196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe288-53cf-44da-a032-c364bb40d21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:13.797563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:13.797933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:13.798311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] PATH_INFO: `/v3/projects/478f04f30cea4a189aa9d9c5d3df376c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:13.799811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478f04f30cea4a189aa9d9c5d3df376c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:13.802143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:13.829322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6dfbe288-53cf-44da-a032-c364bb40d21f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:13.831326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6dfbe288-53cf-44da-a032-c364bb40d21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:13.853640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1268/2512] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:13.861219 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:13.861643 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:13.862021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:13.886197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:13.887657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:13.896930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1269/2513] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:13.901864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:13.914267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49727b9-e707-447b-a390-d57cd42d36ea None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:13.917703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:13.918386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:13.918664 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:13.918966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:13.919680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:13.919977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:13.921693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:13.935718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1270/2514] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:34:13.952138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478f04f30cea4a189aa9d9c5d3df376c', 'request_id': 'req-e15025d7-f68b-4615-aa45-1918a61a6791'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.953971 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:13.959158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478f04f30cea4a189aa9d9c5d3df376c', 'request_id': 'req-e15025d7-f68b-4615-aa45-1918a61a6791'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:13.972808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fef75a-2dc6-4f86-a631-25d4e502a1b5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.024931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478f04f30cea4a189aa9d9c5d3df376c', 'request_id': 'req-e15025d7-f68b-4615-aa45-1918a61a6791'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:14.042492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] The token cache is being invalidate because project 478f04f30cea4a189aa9d9c5d3df376c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:14.042967 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:14.043881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-e15025d7-f68b-4615-aa45-1918a61a6791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:14.045740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1245/2515] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:13 2022] DELETE /identity/v3/projects/478f04f30cea4a189aa9d9c5d3df376c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:14.050032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.050825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.051257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.052096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.052824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03932995-b669-4fac-9689-ad9ff5b3d69e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.053269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03932995-b669-4fac-9689-ad9ff5b3d69e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.058286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03932995-b669-4fac-9689-ad9ff5b3d69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.058523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.058523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.064028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:14.089014 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-03932995-b669-4fac-9689-ad9ff5b3d69e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:14.091088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-03932995-b669-4fac-9689-ad9ff5b3d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:14.107045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1246/2516] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:14.121499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.122130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.122597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.150196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:14.151621 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:14.160432 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1247/2517] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:14.166196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.176455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ff8f1-0992-4106-9009-8208900f315d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.181646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.182328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.182609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.182882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.183754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:14.184059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.185990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.200878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1248/2518] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:34:14.205670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.213247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1271/2519] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:13 2022] POST /identity/v3/projects => generated 383 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:14.214657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.216843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:14.218270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfacbe-e7f7-47d4-84fb-a5945abdcf03 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.269183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:14.299935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff144-762a-4c17-a1e2-a172d4193441 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.307492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.309037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.309325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.309555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.310104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.310222 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.310456 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.310819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.311444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.311876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.314181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.315240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.433172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1249/2520] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:14.437385 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.500369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1272/2521] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:14.504861 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.519107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbe1bb-fafb-4937-870d-96242e4f4d34 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.523590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b02120-1b90-4f3c-8d1a-e66459a0e38b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.524971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.525690 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.526028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.526295 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.527028 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.529158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.531572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.532456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.532938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.533414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.534443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.536583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.549321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1273/2522] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:14.553681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.566247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecff1a-9819-4cda-93da-e4123c62d8d0 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.572099 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.572988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.573280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.573564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.574299 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.576059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.588402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1274/2523] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:14.593300 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.604820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6da02-e532-4994-a389-58ff2986ddcf admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.610028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.610985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.611255 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.611532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] PATH_INFO: `/v3/projects/5d71241b8d39418b9656e97e29f1895a/users/ec947b08d17b410c96effc0d9713cf22/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.631587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1250/2524] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:14.633838 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d71241b8d39418b9656e97e29f1895a, user_id=ec947b08d17b410c96effc0d9713cf22, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.637601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.637834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.650637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8799b-f923-4fc3-bd65-aeaea2b7523d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.655693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.662616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.662616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.662616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.662616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.662616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.671302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1251/2525] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:14.676242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.691716 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07824e6-9502-4f5d-aa52-338cf914d2a0 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.698702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7e832-f367-488d-9c03-878fdbfe97f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.699760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7e832-f367-488d-9c03-878fdbfe97f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.700236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7e832-f367-488d-9c03-878fdbfe97f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.700794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7e832-f367-488d-9c03-878fdbfe97f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.702044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7e832-f367-488d-9c03-878fdbfe97f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.704978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7e832-f367-488d-9c03-878fdbfe97f0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.722683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1275/2526] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:14 2022] PUT /identity/v3/projects/5d71241b8d39418b9656e97e29f1895a/users/ec947b08d17b410c96effc0d9713cf22/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:14.729536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.734403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1252/2527] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:14.735972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.736265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.736571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.774366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] MFA Rules not processed for user `ec947b08d17b410c96effc0d9713cf22`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:14.775613 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:14.796555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:14.822070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca3f0-4023-42e6-b08c-0cd84f4200fa admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.830136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0cd49-eede-4ba3-adf9-3a8a0c139229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.831396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0cd49-eede-4ba3-adf9-3a8a0c139229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.831842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0cd49-eede-4ba3-adf9-3a8a0c139229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.832331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0cd49-eede-4ba3-adf9-3a8a0c139229 admin admin] PATH_INFO: `/v3/projects/bbed931b9e754877b3e081393ca9524b/users/9da28a1307434e58bb2297a69e50ad8f/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.858706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0cd49-eede-4ba3-adf9-3a8a0c139229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbed931b9e754877b3e081393ca9524b, user_id=9da28a1307434e58bb2297a69e50ad8f, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.862338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0cd49-eede-4ba3-adf9-3a8a0c139229 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.866176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1253/2528] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:14.871232 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:14.882650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7fcb4-eb91-4233-a711-4e2802557c8f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:14.887876 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:14.889340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.889634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.889917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.890637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:14.892824 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:14.957641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1276/2529] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:14 2022] PUT /identity/v3/projects/bbed931b9e754877b3e081393ca9524b/users/9da28a1307434e58bb2297a69e50ad8f/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:14.970827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:14.971182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:14.971557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:14.995566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1254/2530] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:15.001147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.008127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] MFA Rules not processed for user `9da28a1307434e58bb2297a69e50ad8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:15.009475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:15.086725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:15.094883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bed76b-502d-4edb-a6f5-15c564e5d747 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.100181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132676da-0557-48db-a7b8-985af177906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.101593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132676da-0557-48db-a7b8-985af177906e admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.101881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132676da-0557-48db-a7b8-985af177906e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.102165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132676da-0557-48db-a7b8-985af177906e admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.103048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132676da-0557-48db-a7b8-985af177906e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.104871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1277/2531] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:15.109590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132676da-0557-48db-a7b8-985af177906e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.110749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.126284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a4919-665d-474f-9be1-e9e030b2ae80 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.131313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.132004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.132312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.132601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.133305 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.135292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.197493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1255/2532] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:14 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:15.205126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132676da-0557-48db-a7b8-985af177906e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.224399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132676da-0557-48db-a7b8-985af177906e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.233071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.238489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.238696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1278/2533] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:15 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:15.238880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.238880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.238880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.239381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.242250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.259381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1256/2534] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:15.269421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.282315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50d7c2-aff2-4524-b2aa-e081abcb0664 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.348546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20ec09-53dc-41f3-9382-f3cab13565a2 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.356463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.357599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.358070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.358554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.359758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.362975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.363121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.364157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.364532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.364863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] PATH_INFO: `/v3/projects/cbc0d91ccdb1429caa02771cb0a65763/users/92400a5f40a54986ba60a87259ba7935/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.387366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc0d91ccdb1429caa02771cb0a65763, user_id=92400a5f40a54986ba60a87259ba7935, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.396675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.511337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1279/2535] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:15 2022] POST /identity/v3/users => generated 299 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:15.516519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.517726 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:15.524714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1257/2536] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:15 2022] PUT /identity/v3/projects/cbc0d91ccdb1429caa02771cb0a65763/users/92400a5f40a54986ba60a87259ba7935/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:15.529512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.531356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ee37a-93ee-4f3e-9151-a0d03191d94c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.624184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.625606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.625966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.626976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.627164 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.629326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.633285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7274e7-87be-45f0-8390-e941714bb0b1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.640832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.641649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.642013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.642353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.643315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.645422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.646623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1280/2537] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:15.651819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.665744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1258/2538] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:15.667264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.667621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8d7be-ffce-4e79-a092-6885be04222e admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.674285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.675705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.675953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.676204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] PATH_INFO: `/v3/projects/67f4c10e5f6d4f1783ca69394cd80bd0/users/19181bc3014a49e9ab48082700465f78/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.680477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b8438-c3df-47b0-8559-da214569adc8 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.688052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838760b-40c0-45ad-bbc7-7b05b913a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.689242 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838760b-40c0-45ad-bbc7-7b05b913a78d admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.689721 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838760b-40c0-45ad-bbc7-7b05b913a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.690324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838760b-40c0-45ad-bbc7-7b05b913a78d admin admin] PATH_INFO: `/v3/projects/cbc0d91ccdb1429caa02771cb0a65763/users/92400a5f40a54986ba60a87259ba7935/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.693327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838760b-40c0-45ad-bbc7-7b05b913a78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc0d91ccdb1429caa02771cb0a65763, user_id=92400a5f40a54986ba60a87259ba7935, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.698046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838760b-40c0-45ad-bbc7-7b05b913a78d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.702171 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f4c10e5f6d4f1783ca69394cd80bd0, user_id=19181bc3014a49e9ab48082700465f78, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.706452 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.713394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1259/2539] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:15 2022] PUT /identity/v3/projects/cbc0d91ccdb1429caa02771cb0a65763/users/92400a5f40a54986ba60a87259ba7935/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:15.725449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.726566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.726970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.759296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] MFA Rules not processed for user `92400a5f40a54986ba60a87259ba7935`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:15.760761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:15.823110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1281/2540] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:15 2022] PUT /identity/v3/projects/67f4c10e5f6d4f1783ca69394cd80bd0/users/19181bc3014a49e9ab48082700465f78/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:15.827922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.882251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1260/2541] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:15.882468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:15.895431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.898443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:15.917234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2a67-914d-4022-9091-d254c3c6bc03 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.925007 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.926168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.926660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.927126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.928429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.932368 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:15.951448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1282/2542] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:15.956810 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:15.973113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41585bd7-e03f-4f7a-a4ba-a47fe83542af admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.982548 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a558e-ecbc-4809-8653-ee730dc08abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:15.984032 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a558e-ecbc-4809-8653-ee730dc08abc admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:15.984529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a558e-ecbc-4809-8653-ee730dc08abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:15.984987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a558e-ecbc-4809-8653-ee730dc08abc admin admin] PATH_INFO: `/v3/projects/67f4c10e5f6d4f1783ca69394cd80bd0/users/19181bc3014a49e9ab48082700465f78/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:15.988803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a558e-ecbc-4809-8653-ee730dc08abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f4c10e5f6d4f1783ca69394cd80bd0, user_id=19181bc3014a49e9ab48082700465f78, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:15.993291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67814c8-5b1a-4416-a72c-84b30cad9e5f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:15.994862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a558e-ecbc-4809-8653-ee730dc08abc admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:16.015314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1283/2543] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:15 2022] PUT /identity/v3/projects/67f4c10e5f6d4f1783ca69394cd80bd0/users/19181bc3014a49e9ab48082700465f78/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:16.018549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:16.028868 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.029281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.029737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.058066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:16.058983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.059458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.059868 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.079339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] MFA Rules not processed for user `19181bc3014a49e9ab48082700465f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:16.081477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:16.138827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:16.140344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:16.153627 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1261/2544] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:16.204471 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1284/2545] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:16.218981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:16.231439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01846048-5f59-4467-b9ee-f2d03d8d852d service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:16.313976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:16.315828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:16.324745 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:16.325887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.326229 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.326610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.339743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:16.341341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:16.354266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1262/2546] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:16.432038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b6e78-2e56-478b-89ac-29a5f990ed53 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:16.432038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:16.432038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.432038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.433027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.454033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:16.526334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:16.527736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:16.532181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:16.533892 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae13795-1bdd-49da-b1f6-ecc2023f89d3 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:16.538744 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:16.539415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1285/2547] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:16.539642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.539931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.540214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] PATH_INFO: `/v3/users/9da28a1307434e58bb2297a69e50ad8f` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.540401 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:16.541596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da28a1307434e58bb2297a69e50ad8f)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:16.541765 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:16.543638 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:16.558272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f4412-3d9a-4e78-ab2b-d386ca4b91f3 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:16.564169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:16.565417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.565708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.565989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] PATH_INFO: `/v3/users/b72b94a289cf462c8a9251e1d2866288` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.567337 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72b94a289cf462c8a9251e1d2866288)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:16.568084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:16.569362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:16.743573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da28a1307434e58bb2297a69e50ad8f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.744425 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da28a1307434e58bb2297a69e50ad8f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.757036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da28a1307434e58bb2297a69e50ad8f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.773802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da28a1307434e58bb2297a69e50ad8f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.774809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-012da125-8f1b-4912-81d4-9e7876150cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da28a1307434e58bb2297a69e50ad8f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.781522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72b94a289cf462c8a9251e1d2866288'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.795308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72b94a289cf462c8a9251e1d2866288'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.796499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72b94a289cf462c8a9251e1d2866288'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.819803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72b94a289cf462c8a9251e1d2866288'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.821003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-589bf7b2-7d25-48c8-936e-febe4e902a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72b94a289cf462c8a9251e1d2866288'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:16.860031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1263/2548] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:16 2022] DELETE /identity/v3/users/9da28a1307434e58bb2297a69e50ad8f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:16.866568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.867040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.867449 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.892262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:16.893924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:16.977760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1286/2549] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:16 2022] DELETE /identity/v3/users/b72b94a289cf462c8a9251e1d2866288 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:16.984574 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:16.985056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:16.985532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:16.991910 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1264/2550] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:17.008847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:17.010325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:17.012096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:17.015054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:17.105297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1287/2551] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:17.117532 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:17.132890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae2da-ef88-4e7b-b630-64288841cd51 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:17.185891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6a815-84aa-4be2-a820-3f93eaff7fa9 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:17.189877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:17.190463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:17.190751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:17.191045 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:17.202320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:17.203780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:17.215327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1288/2552] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:17.219060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:17.219750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:17.220227 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:17.225321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:17.227376 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:17.231519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:17.233183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:17.244367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1265/2553] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:17.815943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:17.817805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:17.835594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cd5c-40d7-4ec7-aae0-71e1949aa91f service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:17.842443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:17.848615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:17.848615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:17.848615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] PATH_INFO: `/v3/projects/d5d890325797406ba7a5bdb263c41713` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:17.848615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d890325797406ba7a5bdb263c41713)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:17.848615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:17.857136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:17.879157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:17.880955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:17.899052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1d037-ae96-4dfd-8079-d43a924fc2f3 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:17.904418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:17.905318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:17.905604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:17.905940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] PATH_INFO: `/v3/projects/bbed931b9e754877b3e081393ca9524b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:17.907280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbed931b9e754877b3e081393ca9524b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:17.910451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:17.981931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d890325797406ba7a5bdb263c41713', 'request_id': 'req-727bfec6-8d49-4bbc-be70-80ddaecc4216'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:17.988876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d890325797406ba7a5bdb263c41713', 'request_id': 'req-727bfec6-8d49-4bbc-be70-80ddaecc4216'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.036836 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbed931b9e754877b3e081393ca9524b', 'request_id': 'req-f86f530b-7d73-4940-9370-c81478f40f45'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.043953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbed931b9e754877b3e081393ca9524b', 'request_id': 'req-f86f530b-7d73-4940-9370-c81478f40f45'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.099302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d890325797406ba7a5bdb263c41713', 'request_id': 'req-727bfec6-8d49-4bbc-be70-80ddaecc4216'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.111984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] The token cache is being invalidate because project d5d890325797406ba7a5bdb263c41713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:18.112336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.113035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-727bfec6-8d49-4bbc-be70-80ddaecc4216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.114647 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1289/2554] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:17 2022] DELETE /identity/v3/projects/d5d890325797406ba7a5bdb263c41713 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:18.120170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97354bd1-afb4-49fc-9af9-5de1b0470627 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.120762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97354bd1-afb4-49fc-9af9-5de1b0470627 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.121053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97354bd1-afb4-49fc-9af9-5de1b0470627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.142357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-97354bd1-afb4-49fc-9af9-5de1b0470627 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:18.143666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-97354bd1-afb4-49fc-9af9-5de1b0470627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:18.209436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1290/2555] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:18.211528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbed931b9e754877b3e081393ca9524b', 'request_id': 'req-f86f530b-7d73-4940-9370-c81478f40f45'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.218780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.219062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.219375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.227383 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] The token cache is being invalidate because project bbed931b9e754877b3e081393ca9524b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:18.227954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.228852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.231984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1266/2556] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:17 2022] DELETE /identity/v3/projects/bbed931b9e754877b3e081393ca9524b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:18.237332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.239712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:18.245505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:18.247503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:18.263190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1291/2557] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:18.263455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f530b-7d73-4940-9370-c81478f40f45 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.267324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.268821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.269605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.269888 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.270328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] PATH_INFO: `/v3/users/19181bc3014a49e9ab48082700465f78` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.271626 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=19181bc3014a49e9ab48082700465f78)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.273652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:18.282230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe633b3-0f2a-4ef2-bf3c-18b43a6ef95d None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.289931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.290968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.291386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.291807 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.293291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:18.293758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.296411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:18.315586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1292/2558] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:34:18.318592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.330699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeb38b-37f7-49a9-b46d-382ee8925064 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.335614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.336315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.336611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.336913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.337649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.339623 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:18.471437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1293/2559] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:18 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:18.476529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.511398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19181bc3014a49e9ab48082700465f78'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.512272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19181bc3014a49e9ab48082700465f78'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.521297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19181bc3014a49e9ab48082700465f78'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.538598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19181bc3014a49e9ab48082700465f78'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.539475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19181bc3014a49e9ab48082700465f78'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:18.554975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:18.560937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb00e1-4741-42d0-9e8c-04cf2eb1b193 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.566126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.566863 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.567148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.567430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.568156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.570331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:18.657931 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1267/2560] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:18 2022] DELETE /identity/v3/users/19181bc3014a49e9ab48082700465f78 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:18.664069 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1294/2561] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:18 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:18.665501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.667333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:18.748080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:18.770592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bf97d-6b3d-4f93-b94d-e884dda30aef admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.776417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.777479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.783688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.784267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.785364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.788025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:18.803140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1268/2562] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:18.807414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.818624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840e3b3-7f87-475c-b4f4-d589f4e0afd1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.823614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bb0d5-504c-447d-a189-e47012c4ae65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.824319 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb0d5-504c-447d-a189-e47012c4ae65 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.824621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb0d5-504c-447d-a189-e47012c4ae65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.824903 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb0d5-504c-447d-a189-e47012c4ae65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.825649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bb0d5-504c-447d-a189-e47012c4ae65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.827360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bb0d5-504c-447d-a189-e47012c4ae65 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:18.839036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1295/2563] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:18.848829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:18.860568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a090425-3d1a-4e3a-a9b1-4116b8f71744 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:18.865412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:18.866367 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:18.866655 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:18.866984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] PATH_INFO: `/v3/projects/a6c54f3c726d412a8f9fd25d84b3743c/users/ad8751a50d574a28b8e92ac5519c673a/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:18.885832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c54f3c726d412a8f9fd25d84b3743c, user_id=ad8751a50d574a28b8e92ac5519c673a, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:18.888600 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.026536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1269/2564] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:18 2022] PUT /identity/v3/projects/a6c54f3c726d412a8f9fd25d84b3743c/users/ad8751a50d574a28b8e92ac5519c673a/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:19.029511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.029993 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.030460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.071654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] MFA Rules not processed for user `ad8751a50d574a28b8e92ac5519c673a`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:19.073703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:19.176134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1296/2565] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:19.181561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.234821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.237408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:19.263669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7628aa4-6e35-4ee8-a2b4-af1021ce6029 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.269794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.270642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.271027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.271414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.272228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.274405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.358674 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b03d1-5125-44f9-9a0f-f73d760888e7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.363557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.364493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.364784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.365067 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] PATH_INFO: `/v3/projects/67f4c10e5f6d4f1783ca69394cd80bd0` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.366889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f4c10e5f6d4f1783ca69394cd80bd0)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.368961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.380976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1270/2566] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:19 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:19.382292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.462810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95adc9-1dae-4f37-a6d7-4c3f00cea626 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.467778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.472399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.472399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.472399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.472399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.472399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.551937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f4c10e5f6d4f1783ca69394cd80bd0', 'request_id': 'req-7c849d81-ec0f-411d-926c-1baad7d221f3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:19.562924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f4c10e5f6d4f1783ca69394cd80bd0', 'request_id': 'req-7c849d81-ec0f-411d-926c-1baad7d221f3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:19.568351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1271/2567] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:19 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:19.572079 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.590108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8599fd-d019-4ec9-80a9-f3aadaacb878 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.640274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:19.666914 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f4c10e5f6d4f1783ca69394cd80bd0', 'request_id': 'req-7c849d81-ec0f-411d-926c-1baad7d221f3'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:19.672793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:19.682271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] The token cache is being invalidate because project 67f4c10e5f6d4f1783ca69394cd80bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:19.684794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:19.684794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7c849d81-ec0f-411d-926c-1baad7d221f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:19.687407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1297/2568] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:19 2022] DELETE /identity/v3/projects/67f4c10e5f6d4f1783ca69394cd80bd0 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:19.693416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70728494-c559-4979-ba55-3a175787dd3c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.693989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70728494-c559-4979-ba55-3a175787dd3c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.695216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70728494-c559-4979-ba55-3a175787dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.696541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.697645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.698213 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.698668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.699993 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.702727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.725610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1272/2569] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:19.727496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.729814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:19.731184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-70728494-c559-4979-ba55-3a175787dd3c None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:19.732537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-70728494-c559-4979-ba55-3a175787dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:19.743375 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1298/2570] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:19.745704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685657b7-4bc2-43f7-90ba-df1578b0d6ea admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.751062 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.752015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.752339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.752603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] PATH_INFO: `/v3/projects/e8aa37d3353147a58f31b9954d39009a/users/a4813bc6d377406eb500ad33e79f46f8/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.753294 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.754250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.754250 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.777368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8aa37d3353147a58f31b9954d39009a, user_id=a4813bc6d377406eb500ad33e79f46f8, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.781779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.783068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:19.785047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:19.799520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1299/2571] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:19.806793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.821787 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815adcdd-d1db-40c7-ac05-e271e01a8cf2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.829877 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.830728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.831165 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.831539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.832477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:19.832849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.834935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.848598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1300/2572] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:34:19.849641 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.851632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:19.866097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e850c25-398c-46d2-8ddb-97706dcebf17 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.872112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.873469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.873932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.874397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] PATH_INFO: `/v3/users/ec947b08d17b410c96effc0d9713cf22` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.876520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec947b08d17b410c96effc0d9713cf22)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.879912 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:19.889520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1273/2573] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:19 2022] PUT /identity/v3/projects/e8aa37d3353147a58f31b9954d39009a/users/a4813bc6d377406eb500ad33e79f46f8/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:19.889520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:19.977225 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091da62c-ba2a-413f-a0f3-3bd93000d453 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:19.982519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:19.983325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:19.983751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:19.984074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:19.984939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:19.987063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.098741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec947b08d17b410c96effc0d9713cf22'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:20.107391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1274/2574] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:19 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:20.109032 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.120597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec947b08d17b410c96effc0d9713cf22'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:20.121801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec947b08d17b410c96effc0d9713cf22'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:20.146747 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec947b08d17b410c96effc0d9713cf22'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:20.152365 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec947b08d17b410c96effc0d9713cf22'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:20.229021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45722a1e-11f6-47b7-8940-bafdc2d5150e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.237337 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0bdc7-b6e6-4618-8d9b-61cf48a44a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.238496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0bdc7-b6e6-4618-8d9b-61cf48a44a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.239012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0bdc7-b6e6-4618-8d9b-61cf48a44a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.239527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0bdc7-b6e6-4618-8d9b-61cf48a44a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.240995 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0bdc7-b6e6-4618-8d9b-61cf48a44a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.245167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0bdc7-b6e6-4618-8d9b-61cf48a44a59 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.251061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1301/2575] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:19 2022] DELETE /identity/v3/users/ec947b08d17b410c96effc0d9713cf22 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:20.255084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.256686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:20.263473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1275/2576] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:20.271310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.272336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.272336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.303579 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:20.305266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:20.347478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:20.375472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e560c8-74a3-4ba3-807b-80b2c75755c9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.381948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10b7e0-0bd0-4750-bc64-b420cb4d2067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.383135 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10b7e0-0bd0-4750-bc64-b420cb4d2067 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.383670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10b7e0-0bd0-4750-bc64-b420cb4d2067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.384156 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10b7e0-0bd0-4750-bc64-b420cb4d2067 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.385511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10b7e0-0bd0-4750-bc64-b420cb4d2067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.389445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10b7e0-0bd0-4750-bc64-b420cb4d2067 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.400723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1276/2577] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:20.402139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.404300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:20.420646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a743e7-9be3-4f78-bfaa-dfe12d92f47f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.426048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.427080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.427453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.427895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] PATH_INFO: `/v3/projects/e8aa37d3353147a58f31b9954d39009a/users/a4813bc6d377406eb500ad33e79f46f8/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.430412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8aa37d3353147a58f31b9954d39009a, user_id=a4813bc6d377406eb500ad33e79f46f8, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.433264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.457176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1277/2578] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:20 2022] PUT /identity/v3/projects/e8aa37d3353147a58f31b9954d39009a/users/a4813bc6d377406eb500ad33e79f46f8/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:20.457176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.457176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:20.531104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2f697-9b64-4a5a-882a-f961dfd6e8c2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.535943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.536752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.537239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.537424 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.544004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1302/2579] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:20 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:20.546500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.547060 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.547540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.591954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] MFA Rules not processed for user `a4813bc6d377406eb500ad33e79f46f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:20.594450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:20.625090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:20.653700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.655833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.673557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1278/2580] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:20.675036 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.691989 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226a074-ee57-416b-8720-1e440d413ad1 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.700174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.701314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.701811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.702293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.703670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.706711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.734097 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1279/2581] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:20.734608 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1303/2582] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:20.735869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.753458 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.755236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:20.757620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98761c2-de88-4a36-b7a1-2588d15413c5 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.769808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.771807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.772419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.773070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.774302 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.777412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.800365 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1280/2583] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:20.806570 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:20.826630 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba38489-0141-460f-8da0-86ce878b8911 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.835697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.837306 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.837841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.838506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] PATH_INFO: `/v3/projects/5bee02e02b5442d7a801109ceeb349d3/users/23bf4abb51dc47839dfcd10e9522006b/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.851306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054861a-82ef-44b7-8eee-00dd7ef3b4e0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:20.856932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aae3e-ece7-4c73-bccc-6f4348670478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:20.857867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aae3e-ece7-4c73-bccc-6f4348670478 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:20.858349 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aae3e-ece7-4c73-bccc-6f4348670478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:20.858675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aae3e-ece7-4c73-bccc-6f4348670478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:20.862483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bee02e02b5442d7a801109ceeb349d3, user_id=23bf4abb51dc47839dfcd10e9522006b, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.868405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.875408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aae3e-ece7-4c73-bccc-6f4348670478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:20.877035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aae3e-ece7-4c73-bccc-6f4348670478 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:20.889925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1304/2584] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:21.008125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1281/2585] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:20 2022] PUT /identity/v3/projects/5bee02e02b5442d7a801109ceeb349d3/users/23bf4abb51dc47839dfcd10e9522006b/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:21.024855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.025611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.038885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.073084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] MFA Rules not processed for user `23bf4abb51dc47839dfcd10e9522006b`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:21.075297 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:21.172758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1305/2586] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:21.177820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.202528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:21.259408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.261079 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:21.278477 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8b8cf-fafd-4265-b5fa-683e5caafd84 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:21.284751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:21.285851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.286238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.286622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.287560 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:21.292390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:21.292670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c874b5a-4aec-4cc6-9802-6b14e20ae9b4 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:21.312928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:21.316216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.316752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.318050 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] PATH_INFO: `/v3/projects/5d71241b8d39418b9656e97e29f1895a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.320871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d71241b8d39418b9656e97e29f1895a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:21.323430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:21.424767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1282/2587] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:21 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:21.425684 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.484911 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:21.513624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d71241b8d39418b9656e97e29f1895a', 'request_id': 'req-1e32fe74-3559-44c8-aa42-576db29bdd61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.529330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d71241b8d39418b9656e97e29f1895a', 'request_id': 'req-1e32fe74-3559-44c8-aa42-576db29bdd61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.535926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c9d32-4a39-4885-83a0-e33d7ec30326 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:21.601179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d71241b8d39418b9656e97e29f1895a', 'request_id': 'req-1e32fe74-3559-44c8-aa42-576db29bdd61'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.612124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:21.613338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.613883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.614351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.615536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:21.619057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:21.623245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] The token cache is being invalidate because project 5d71241b8d39418b9656e97e29f1895a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:21.623704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.624698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.627596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1306/2588] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:21 2022] DELETE /identity/v3/projects/5d71241b8d39418b9656e97e29f1895a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:21.635969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.638563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:21.657954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32fe74-3559-44c8-aa42-576db29bdd61 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:21.666957 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:21.668593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.669057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.669515 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] PATH_INFO: `/v3/users/92400a5f40a54986ba60a87259ba7935` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.671809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=92400a5f40a54986ba60a87259ba7935)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:21.675428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:21.728327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1283/2589] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:21 2022] POST /identity/v3/users => generated 299 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:21.733059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.735597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:21.751733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9e97a-804a-44d0-81d3-155404ab20d0 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:21.758677 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:21.759494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.764987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.764987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.764987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:21.764987 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:21.777175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1284/2590] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:21.783282 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.797043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331efde5-048a-4fad-8cc5-bd4b2fe11fa0 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:21.802945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:21.804490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:21.804774 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:21.805094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] PATH_INFO: `/v3/projects/ba77c2893f45483ca902169147156c6c/users/f6d8033fbada4360a51833819263a55b/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:21.828398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba77c2893f45483ca902169147156c6c, user_id=f6d8033fbada4360a51833819263a55b, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:21.833500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:21.916483 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92400a5f40a54986ba60a87259ba7935'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.926075 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92400a5f40a54986ba60a87259ba7935'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.927174 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92400a5f40a54986ba60a87259ba7935'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.948659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92400a5f40a54986ba60a87259ba7935'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.949952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92400a5f40a54986ba60a87259ba7935'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:21.950587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1285/2591] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:21 2022] PUT /identity/v3/projects/ba77c2893f45483ca902169147156c6c/users/f6d8033fbada4360a51833819263a55b/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:21.952548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:21.955571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:21.974253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:22.021211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1307/2592] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:21 2022] DELETE /identity/v3/users/92400a5f40a54986ba60a87259ba7935 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:22.061999 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bb0e-11a3-4810-8347-678bf797ef0d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:22.137137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7be36-3a8e-4194-b3d4-ba7fc05df9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:22.139925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7be36-3a8e-4194-b3d4-ba7fc05df9dc admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.140797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7be36-3a8e-4194-b3d4-ba7fc05df9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.144388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7be36-3a8e-4194-b3d4-ba7fc05df9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.144388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7be36-3a8e-4194-b3d4-ba7fc05df9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:22.153606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7be36-3a8e-4194-b3d4-ba7fc05df9dc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:22.172732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1286/2593] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:22.177061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:22.193121 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ff761-3466-40ac-9dc0-94721056b81f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:22.201228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:22.202784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.203277 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.203768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] PATH_INFO: `/v3/projects/ba77c2893f45483ca902169147156c6c/users/f6d8033fbada4360a51833819263a55b/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.207586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba77c2893f45483ca902169147156c6c, user_id=f6d8033fbada4360a51833819263a55b, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:22.212639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:22.232190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1308/2594] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:22 2022] PUT /identity/v3/projects/ba77c2893f45483ca902169147156c6c/users/f6d8033fbada4360a51833819263a55b/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:22.251710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.252216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.252754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.261421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:22.263571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.296796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] MFA Rules not processed for user `f6d8033fbada4360a51833819263a55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:22.298829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.359085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e23899-1f59-4b25-9afa-184ba75047f8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:22.365435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:22.367106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.367494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.368533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] PATH_INFO: `/v3/users/ad8751a50d574a28b8e92ac5519c673a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.369516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8751a50d574a28b8e92ac5519c673a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:22.372037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:22.440018 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1287/2595] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:22.449013 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:22.451434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.527938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83e746-9f8e-427c-acfe-d2924bdc6de9 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:22.532577 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:22.533253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.533605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.533865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.535921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.554320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:22.610826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:22.612704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:22.623822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1288/2596] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:22.658357 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:22.674066 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8751a50d574a28b8e92ac5519c673a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.683571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8751a50d574a28b8e92ac5519c673a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.684793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8751a50d574a28b8e92ac5519c673a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.703293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8751a50d574a28b8e92ac5519c673a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.704921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-48f513e3-a166-4b83-a848-adf44edcb2fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8751a50d574a28b8e92ac5519c673a'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.712674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:22.714965 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.744595 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73945-825c-4102-8c27-e75bf3bfb72a service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:22.753091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:22.755532 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.756023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.756547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] PATH_INFO: `/v3/projects/cbc0d91ccdb1429caa02771cb0a65763` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.759172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc0d91ccdb1429caa02771cb0a65763)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:22.762884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:22.829207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1309/2597] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:22 2022] DELETE /identity/v3/users/ad8751a50d574a28b8e92ac5519c673a => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:22.836559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:22.837035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:22.837508 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:22.871215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:22.873127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.927247 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc0d91ccdb1429caa02771cb0a65763', 'request_id': 'req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.935493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc0d91ccdb1429caa02771cb0a65763', 'request_id': 'req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:22.955854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1310/2598] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:22.966424 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:22.967847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:22.996597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc0d91ccdb1429caa02771cb0a65763', 'request_id': 'req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:23.016743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] The token cache is being invalidate because project cbc0d91ccdb1429caa02771cb0a65763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:23.017284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:23.021206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9d957d2b-9f6d-47c9-babe-da17c7ee2f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:23.024169 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1289/2599] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:22 2022] DELETE /identity/v3/projects/cbc0d91ccdb1429caa02771cb0a65763 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:23.030399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597531f-be9c-4ce2-bca7-73d4b05c441d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.030773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597531f-be9c-4ce2-bca7-73d4b05c441d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.031177 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597531f-be9c-4ce2-bca7-73d4b05c441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.044738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeb181-3199-4780-9e7b-9b2d67357076 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.054124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d597531f-be9c-4ce2-bca7-73d4b05c441d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:23.062179 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d597531f-be9c-4ce2-bca7-73d4b05c441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:23.116617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.116617 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.116929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.117089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.119160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:23.168025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1290/2600] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:23.177140 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.177573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.178983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.205102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.207567 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.207758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:23.209366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:23.218915 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1311/2601] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:23.226453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1291/2602] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:23.227652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.239212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b388b-de3a-4a15-a611-4c4c0c3a1212 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.244956 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.245771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.246227 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.246631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.247484 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:23.247893 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.249796 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.283183 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1312/2603] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:34:23.284858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.302902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594258b-07c3-4358-a17c-ea5e90f76912 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.314000 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.315708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.316214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.317186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.318525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.322184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.328266 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:23.506148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:23.514968 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1292/2604] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:23 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:23.519620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.531758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bb8b3-b923-4f68-a8d5-691df0dc77d1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.635412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.636650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.637117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.643850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.643850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.643850 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.767338 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1313/2605] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:23 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:23.772593 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.787568 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585544a8-0d39-47f8-9002-9ea2472ea7bd admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.796754 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.798607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.799669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.801297 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.803076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.807815 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.821835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1293/2606] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:23.827113 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.839038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5d88-5a15-485e-845b-949ea229f2b7 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.844654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.845373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.845793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.845938 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.846652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.848315 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.861733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1314/2607] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:23.868278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.888800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a6ad8-25f1-4861-8e2c-b8efc63bf44d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.900010 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:23.902111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941a14-2bb4-4e60-85ef-283b78b1401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.903148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:23.903664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941a14-2bb4-4e60-85ef-283b78b1401a admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.904119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941a14-2bb4-4e60-85ef-283b78b1401a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.904702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941a14-2bb4-4e60-85ef-283b78b1401a admin admin] PATH_INFO: `/v3/projects/40de23de45c94d17b950b0b25fd8eb8d/users/e4ebacfba9fb419fba6cd53a7f388a41/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.927640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0cb1d-a7f0-4a30-91a7-0fa53c0bdb48 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:23.934381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941a14-2bb4-4e60-85ef-283b78b1401a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40de23de45c94d17b950b0b25fd8eb8d, user_id=e4ebacfba9fb419fba6cd53a7f388a41, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.936441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:23.938071 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:23.938633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:23.938976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] PATH_INFO: `/v3/projects/a6c54f3c726d412a8f9fd25d84b3743c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:23.939414 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941a14-2bb4-4e60-85ef-283b78b1401a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:23.941190 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c54f3c726d412a8f9fd25d84b3743c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:23.944985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:24.019524 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1294/2608] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:23 2022] PUT /identity/v3/projects/40de23de45c94d17b950b0b25fd8eb8d/users/e4ebacfba9fb419fba6cd53a7f388a41/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:24.033189 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:24.033529 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:24.033863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:24.066857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] MFA Rules not processed for user `e4ebacfba9fb419fba6cd53a7f388a41`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:24.068499 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:24.097143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c54f3c726d412a8f9fd25d84b3743c', 'request_id': 'req-e9f89403-2ad5-4e67-a929-04a3e237da50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.108431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c54f3c726d412a8f9fd25d84b3743c', 'request_id': 'req-e9f89403-2ad5-4e67-a929-04a3e237da50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.155723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1295/2609] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:24.160720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:24.195076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c54f3c726d412a8f9fd25d84b3743c', 'request_id': 'req-e9f89403-2ad5-4e67-a929-04a3e237da50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.217905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] The token cache is being invalidate because project a6c54f3c726d412a8f9fd25d84b3743c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:24.218613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.219612 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.222000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1315/2610] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:23 2022] DELETE /identity/v3/projects/a6c54f3c726d412a8f9fd25d84b3743c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:24.228377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:24.230606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:24.281670 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea2db1-56bf-43ac-932a-188f3fabcfea None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:24.335556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89403-2ad5-4e67-a929-04a3e237da50 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:24.343210 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:24.344926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:24.345392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:24.345855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] PATH_INFO: `/v3/users/a4813bc6d377406eb500ad33e79f46f8` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:24.347945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4813bc6d377406eb500ad33e79f46f8)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:24.351781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:24.383052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:24.383912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:24.384184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:24.384625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:24.385415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:24.389219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:24.455767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:24.551025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:24.567742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1296/2611] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:24 2022] POST /identity/v3/projects => generated 387 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:24.569519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:24.571555 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:24.648396 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4813bc6d377406eb500ad33e79f46f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.660621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4813bc6d377406eb500ad33e79f46f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.661499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4813bc6d377406eb500ad33e79f46f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.663913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86a006-7b5f-40b8-8307-b775acb9dcda admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:24.671258 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:24.672389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:24.672946 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:24.673426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:24.675395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:24.680086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:24.680376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4813bc6d377406eb500ad33e79f46f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.687090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4813bc6d377406eb500ad33e79f46f8'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:24.863625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1297/2612] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:24 2022] POST /identity/v3/users => generated 300 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:24.870607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1316/2613] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:24 2022] DELETE /identity/v3/users/a4813bc6d377406eb500ad33e79f46f8 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:24.873688 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:24.877196 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:24.902751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7726-54d0-49b6-bee1-c79bc9004e94 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:24.997813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:24.999806 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.001233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.002128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.005344 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:25.009566 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:25.031622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1298/2614] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:25.039486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:25.056896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d5701-0600-4fa2-b7a5-1074720956a8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:25.274422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:25.274422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.274422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.274422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] PATH_INFO: `/v3/projects/688bd9f9113b49979525926e2afc19da/users/876c8b7b6b8d42de85cf9add0d3e6ed4/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.276149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688bd9f9113b49979525926e2afc19da, user_id=876c8b7b6b8d42de85cf9add0d3e6ed4, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:25.276149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:25.459847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1317/2615] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:25 2022] PUT /identity/v3/projects/688bd9f9113b49979525926e2afc19da/users/876c8b7b6b8d42de85cf9add0d3e6ed4/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:25.467038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:25.488873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a34bce-6a21-4ce8-8795-067b750070c1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:25.591682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:25.599415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:25.600404 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.600917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.601793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.602834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:25.604860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:25.620944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1299/2616] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:25.625702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:25.639127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b41efc-4450-4c0d-a469-acd58e052f47 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:25.644973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77314bcd-fe55-48b6-87c9-796b8ba6571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:25.646248 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314bcd-fe55-48b6-87c9-796b8ba6571d admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.646780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314bcd-fe55-48b6-87c9-796b8ba6571d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.647703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314bcd-fe55-48b6-87c9-796b8ba6571d admin admin] PATH_INFO: `/v3/projects/688bd9f9113b49979525926e2afc19da/users/876c8b7b6b8d42de85cf9add0d3e6ed4/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.650111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77314bcd-fe55-48b6-87c9-796b8ba6571d admin admin] RBAC: Authorizing `identity:create_grant(project_id=688bd9f9113b49979525926e2afc19da, user_id=876c8b7b6b8d42de85cf9add0d3e6ed4, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:25.653968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77314bcd-fe55-48b6-87c9-796b8ba6571d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:25.658109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:25.660731 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:25.668987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1318/2617] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:25 2022] PUT /identity/v3/projects/688bd9f9113b49979525926e2afc19da/users/876c8b7b6b8d42de85cf9add0d3e6ed4/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:25.687789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.688102 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4d7b-0cd6-41d8-b55f-fbd1a774fe2a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:25.688330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.688634 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.744981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] MFA Rules not processed for user `876c8b7b6b8d42de85cf9add0d3e6ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:25.747662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:25.797004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:25.798521 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.798880 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.799228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] PATH_INFO: `/v3/users/1101e2cdc1074373bbb8c9119974b075` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.801251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1101e2cdc1074373bbb8c9119974b075)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:25.805849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:25.855949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:25.877082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1319/2618] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:25.878333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:25.880276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:25.907596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd467b43-d289-47f5-83fc-e7d51a1b6749 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:25.913987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:25.915322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:25.915782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:25.916136 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] PATH_INFO: `/v3/projects/e8aa37d3353147a58f31b9954d39009a` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:25.918225 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8aa37d3353147a58f31b9954d39009a)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:25.921081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:26.024462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1101e2cdc1074373bbb8c9119974b075'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.025578 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1101e2cdc1074373bbb8c9119974b075'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.049166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1101e2cdc1074373bbb8c9119974b075'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.054813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8aa37d3353147a58f31b9954d39009a', 'request_id': 'req-3e0af5f7-f967-4e44-a300-7d150f1e4b54'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.065082 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8aa37d3353147a58f31b9954d39009a', 'request_id': 'req-3e0af5f7-f967-4e44-a300-7d150f1e4b54'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.065379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1101e2cdc1074373bbb8c9119974b075'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.066154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1101e2cdc1074373bbb8c9119974b075'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.191447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1300/2619] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:25 2022] DELETE /identity/v3/users/1101e2cdc1074373bbb8c9119974b075 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:26.192417 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:26.193846 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:26.262399 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4533-f7dc-4430-8257-f35c66569249 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:26.267635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:26.268728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.269128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.269528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.271826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:26.303738 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8aa37d3353147a58f31b9954d39009a', 'request_id': 'req-3e0af5f7-f967-4e44-a300-7d150f1e4b54'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.325597 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] The token cache is being invalidate because project e8aa37d3353147a58f31b9954d39009a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:26.335767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.335767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3e0af5f7-f967-4e44-a300-7d150f1e4b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:26.335767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1320/2620] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:25 2022] DELETE /identity/v3/projects/e8aa37d3353147a58f31b9954d39009a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:26.335767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.335767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.335767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.353344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:26.355073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:26.375634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:26.377720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9187a-6bca-4d02-b717-fa7a56e70821 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:26.395113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1301/2621] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:26.396924 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3cf6f-684d-4165-9a38-6f7a4c48ea06 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.397829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3cf6f-684d-4165-9a38-6f7a4c48ea06 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.398214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3cf6f-684d-4165-9a38-6f7a4c48ea06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.426545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-2ae3cf6f-684d-4165-9a38-6f7a4c48ea06 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:26.428704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-2ae3cf6f-684d-4165-9a38-6f7a4c48ea06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:26.444734 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:26.448715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1321/2622] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:26.461656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:26.463642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:26.509415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1302/2623] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:26.523592 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.524104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.524793 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.544772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a315b-16fe-416e-8306-95dbb8918542 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:26.548887 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:26.549551 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.550173 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.550596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.554202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:26.556838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:26.561943 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:26.563691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:26.568654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1303/2624] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:26.575184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:26.575506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1322/2625] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:26.589940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5150-4b85-43aa-92c4-96ec2e7e274f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:26.599783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:26.601553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.602083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.602601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.604087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:26.604689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:26.607928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:26.629863 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1304/2626] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:34:26.636867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:26.654348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2a768-8b51-4f5b-9fee-f8a334409d7f service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:26.665177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:26.665935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.666224 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.683257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.683257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:26.683257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:26.790615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1323/2627] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:26 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:26.796893 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:26.814527 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f94e3-c7bf-4866-9edd-8e91ebaebe14 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:26.923075 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:26.924272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:26.933053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:26.933053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:26.933053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:26.933053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.031763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1305/2628] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:26 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:27.036797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.050222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a39611-294c-4f98-9f20-ebca454f5d2b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.058284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.058284 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.058632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.059085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.060354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.063124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.082110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1324/2629] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:27.087911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.102652 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e7d93-815a-4673-8e0d-c4e4bcaf34b7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.111081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.112323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.112852 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.113318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.114772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.117795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.134084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1306/2630] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:27.150711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.170330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286a7c1-9d7c-4db3-bdde-9fff2a78ccc1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.179803 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05966-3ba7-4d75-9c2c-e8077964b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.181356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05966-3ba7-4d75-9c2c-e8077964b948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.181826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05966-3ba7-4d75-9c2c-e8077964b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.182292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05966-3ba7-4d75-9c2c-e8077964b948 admin admin] PATH_INFO: `/v3/projects/08358393d5c5450089399204a38a5035/users/a7e576a30ce441c38fe78b41a8b90b9c/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.208726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05966-3ba7-4d75-9c2c-e8077964b948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08358393d5c5450089399204a38a5035, user_id=a7e576a30ce441c38fe78b41a8b90b9c, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.208726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05966-3ba7-4d75-9c2c-e8077964b948 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.246635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.249773 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:27.283329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470324-24aa-4173-b729-6f718f6ef7e9 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.292273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.293335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.293712 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.294087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] PATH_INFO: `/v3/projects/7d257757a602463089cc5cb6e962ebc9` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.296069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d257757a602463089cc5cb6e962ebc9)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.298292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.337334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1325/2631] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:27 2022] PUT /identity/v3/projects/08358393d5c5450089399204a38a5035/users/a7e576a30ce441c38fe78b41a8b90b9c/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:27.345761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.346070 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.346321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.390000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] MFA Rules not processed for user `a7e576a30ce441c38fe78b41a8b90b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:27.392846 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:27.429927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d257757a602463089cc5cb6e962ebc9', 'request_id': 'req-a306d8d0-c768-485d-8c2a-210c3a31f361'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:27.433140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:27.437862 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d257757a602463089cc5cb6e962ebc9', 'request_id': 'req-a306d8d0-c768-485d-8c2a-210c3a31f361'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:27.477202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1326/2632] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:27.482040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.539452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d257757a602463089cc5cb6e962ebc9', 'request_id': 'req-a306d8d0-c768-485d-8c2a-210c3a31f361'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:27.554458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] The token cache is being invalidate because project 7d257757a602463089cc5cb6e962ebc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:27.554945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:27.555988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a306d8d0-c768-485d-8c2a-210c3a31f361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:27.558103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1307/2633] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:27 2022] DELETE /identity/v3/projects/7d257757a602463089cc5cb6e962ebc9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:27.564483 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec5251-9161-414d-bdcf-631f116389ed None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.565020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec5251-9161-414d-bdcf-631f116389ed None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.565646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec5251-9161-414d-bdcf-631f116389ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.573312 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0869639-920a-4c17-9048-ef7f8c8c682f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.595539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-efec5251-9161-414d-bdcf-631f116389ed None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:27.596752 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-efec5251-9161-414d-bdcf-631f116389ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:27.654259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.655399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.655872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.656472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.657506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.660892 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.683381 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1308/2634] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:27.694398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.694830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.695360 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.718738 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:27.720192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:27.729691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1309/2635] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:27.735066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.749430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5ac79-e4e0-4be0-8514-f1cc4c91aad9 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.754711 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.755391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.755830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.756124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.756951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:27.757270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.758886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.771925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1310/2636] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:34:27.777304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.788402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3e75c-14e6-45e2-9f74-f1401ea2cc16 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.793352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.794088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.794430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.794707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.795434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.797436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.844313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1327/2637] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:27 2022] POST /identity/v3/projects => generated 377 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:27.848931 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.851422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:27.931500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e473e-99b7-4be1-9563-7ceaa2321b82 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:27.936627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:27.937335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:27.937630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:27.937909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:27.938627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:27.940616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:27.949933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1311/2638] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:27 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:27.949933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:27.949933 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:27.972181 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:28.027969 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76c210-74b7-4084-af6b-46aa26ccf22a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.033099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.034450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.034450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.034803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] PATH_INFO: `/v3/users/e4ebacfba9fb419fba6cd53a7f388a41` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.036103 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ebacfba9fb419fba6cd53a7f388a41)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.038708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.050030 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1328/2639] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:27 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:28.051038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.062561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2a97-16c8-417d-9c7b-2b00e6d6460d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.069624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.070385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.070666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.070942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.071711 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.073832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.174139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1329/2640] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:28 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:28.174139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.183393 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3ede-6fee-4a60-9529-8314cf9325f4 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.189033 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.190236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.190774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.191280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.192941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.196126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.210578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1330/2641] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:28.212090 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.224812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02794d-9d25-4067-a3cb-59f8a0d75dde admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.230656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.231464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.231794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.232279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.233453 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.235260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.250087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1331/2642] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:28.251314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.265139 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2990db-44e3-4ee0-81b0-078bed499d98 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.272428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.274119 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.274402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.275411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] PATH_INFO: `/v3/projects/4689727bce874b83a02ed9e7f34e417b/users/c688d41577204551bd046775bc864d50/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.299378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4689727bce874b83a02ed9e7f34e417b, user_id=c688d41577204551bd046775bc864d50, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.302057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ebacfba9fb419fba6cd53a7f388a41'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:28.303269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ebacfba9fb419fba6cd53a7f388a41'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:28.305537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.316998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ebacfba9fb419fba6cd53a7f388a41'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:28.333857 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ebacfba9fb419fba6cd53a7f388a41'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:28.334795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ebacfba9fb419fba6cd53a7f388a41'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:28.425955 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1332/2643] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:28 2022] PUT /identity/v3/projects/4689727bce874b83a02ed9e7f34e417b/users/c688d41577204551bd046775bc864d50/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:28.427117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.427608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1312/2644] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:27 2022] DELETE /identity/v3/users/e4ebacfba9fb419fba6cd53a7f388a41 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:28.429715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.434720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.436448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.517144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59fd1-3624-4066-bfdf-b46bdb560d48 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.520549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79234d7-6c08-4d45-b719-a640932236a0 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.523054 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8812a1-e57d-416e-9a06-63e55317e66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.523749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8812a1-e57d-416e-9a06-63e55317e66c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.524039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8812a1-e57d-416e-9a06-63e55317e66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.524336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8812a1-e57d-416e-9a06-63e55317e66c admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.530256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8812a1-e57d-416e-9a06-63e55317e66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.530256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8812a1-e57d-416e-9a06-63e55317e66c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.530700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.530700 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.531082 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.531583 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.533033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.536183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.539422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1333/2645] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:28.540950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.541236 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.541537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.555730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1313/2646] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:28.555730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.566334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35afa-d76f-4802-959b-46aac6c9452c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.568568 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:28.569988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.573312 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83138883-18a6-4de5-b3d6-033196cf011a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.574832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83138883-18a6-4de5-b3d6-033196cf011a admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.575317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83138883-18a6-4de5-b3d6-033196cf011a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.575757 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83138883-18a6-4de5-b3d6-033196cf011a admin admin] PATH_INFO: `/v3/projects/d7bc412a2d2f4fc68c2a9f3562232807/users/79755c3f9eec4c5ab3e98038015bff32/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.582640 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1334/2647] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:28.590691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.590691 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:28.602756 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83138883-18a6-4de5-b3d6-033196cf011a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bc412a2d2f4fc68c2a9f3562232807, user_id=79755c3f9eec4c5ab3e98038015bff32, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.605597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83138883-18a6-4de5-b3d6-033196cf011a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.605809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625181a1-e4d3-41ac-b7b5-dde788623dad None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.610825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:28.613460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.613460 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.613827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.614056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] PATH_INFO: `/v3/projects/4689727bce874b83a02ed9e7f34e417b/users/c688d41577204551bd046775bc864d50/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.617672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4689727bce874b83a02ed9e7f34e417b, user_id=c688d41577204551bd046775bc864d50, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.622607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.638444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1335/2648] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:28 2022] PUT /identity/v3/projects/4689727bce874b83a02ed9e7f34e417b/users/c688d41577204551bd046775bc864d50/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:28.639480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.641291 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.714442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d86a5-3203-47db-b980-1f466f51d27d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.720411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1314/2649] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:28 2022] PUT /identity/v3/projects/d7bc412a2d2f4fc68c2a9f3562232807/users/79755c3f9eec4c5ab3e98038015bff32/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:28.722260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16fff0-1b9e-4eb0-8984-f2c51cce5d04 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.722618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16fff0-1b9e-4eb0-8984-f2c51cce5d04 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.722978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16fff0-1b9e-4eb0-8984-f2c51cce5d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.766970 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fc16fff0-1b9e-4eb0-8984-f2c51cce5d04 None None] MFA Rules not processed for user `c688d41577204551bd046775bc864d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:28.769270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fc16fff0-1b9e-4eb0-8984-f2c51cce5d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.792918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.793619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.793909 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.794188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.875020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1315/2650] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:28.881094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.881094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.881094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.892195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:28.894916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:28.907536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:28.909755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.910002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1336/2651] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:28.910806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.912439 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.923331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1316/2652] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:28.939254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:28.943625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:28.970928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a776624-6108-4722-84c7-a5b44bb8e18f None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.979452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.983426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.983426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.983426 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.985505 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4481f9e-79dd-4c75-bd41-fb1b21b2b1a7 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:28.990238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:28.990951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:28.991245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:28.991501 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:28.994181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:29.001400 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:29.004775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:29.024572 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1317/2653] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:29.074423 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:29.076245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:29.085492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:29.094958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1337/2654] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:29.216692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:29.220719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:29.246733 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce80807-5b69-41e3-aa2f-2205aefb8560 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:29.254048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:29.255274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:29.255960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:29.256274 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:29.260955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:29.383281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:29.387107 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:29.405505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1318/2655] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:29.466842 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:29.535038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:29.537734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:29.558667 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:29.571887 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1f25a-c375-4432-91d1-fff23a38c7b1 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:29.580773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:29.582836 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:29.583649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:29.584464 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] PATH_INFO: `/v3/projects/40de23de45c94d17b950b0b25fd8eb8d` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:29.592059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40de23de45c94d17b950b0b25fd8eb8d)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:29.600981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:29.756573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40de23de45c94d17b950b0b25fd8eb8d', 'request_id': 'req-8785583c-a85e-488b-9eaf-6b05b2427557'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:29.769705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40de23de45c94d17b950b0b25fd8eb8d', 'request_id': 'req-8785583c-a85e-488b-9eaf-6b05b2427557'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:29.962785 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40de23de45c94d17b950b0b25fd8eb8d', 'request_id': 'req-8785583c-a85e-488b-9eaf-6b05b2427557'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:29.982075 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] The token cache is being invalidate because project 40de23de45c94d17b950b0b25fd8eb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:29.982619 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:29.983435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:29.989937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1338/2656] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:29 2022] DELETE /identity/v3/projects/40de23de45c94d17b950b0b25fd8eb8d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:29.992604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:30.009596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e373f-f4f9-4b28-b070-4f854fc66840 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:30.090729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:30.095191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:30.095191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:30.095191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] PATH_INFO: `/v3/users/876c8b7b6b8d42de85cf9add0d3e6ed4` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:30.095191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=876c8b7b6b8d42de85cf9add0d3e6ed4)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:30.096727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:30.427159 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876c8b7b6b8d42de85cf9add0d3e6ed4'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:30.428352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876c8b7b6b8d42de85cf9add0d3e6ed4'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:30.441761 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876c8b7b6b8d42de85cf9add0d3e6ed4'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:30.462783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876c8b7b6b8d42de85cf9add0d3e6ed4'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:30.463860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-ebffcf94-65dd-4f56-878c-fa4f4d99c09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876c8b7b6b8d42de85cf9add0d3e6ed4'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:30.640345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1319/2657] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:29 2022] DELETE /identity/v3/users/876c8b7b6b8d42de85cf9add0d3e6ed4 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:31.207149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:31.211148 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:31.224540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:31.243047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785583c-a85e-488b-9eaf-6b05b2427557 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:31.374529 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:31.375417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:31.375688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:31.376146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] PATH_INFO: `/v3/projects/688bd9f9113b49979525926e2afc19da` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:31.377842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688bd9f9113b49979525926e2afc19da)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:31.379821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:31.391580 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:31.626724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688bd9f9113b49979525926e2afc19da', 'request_id': 'req-b918f48c-c4ab-4aa3-8d62-173654323f17'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:31.637374 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688bd9f9113b49979525926e2afc19da', 'request_id': 'req-b918f48c-c4ab-4aa3-8d62-173654323f17'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:31.820495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688bd9f9113b49979525926e2afc19da', 'request_id': 'req-b918f48c-c4ab-4aa3-8d62-173654323f17'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:31.838381 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] The token cache is being invalidate because project 688bd9f9113b49979525926e2afc19da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:31.838939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:31.839883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b918f48c-c4ab-4aa3-8d62-173654323f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:31.842109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1339/2658] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:31 2022] DELETE /identity/v3/projects/688bd9f9113b49979525926e2afc19da => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:31.849642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:31.854202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:31.854202 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:31.884208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:31.885693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:31.984540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1320/2659] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:31.987110 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:31.987536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:31.987927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.018436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:32.020913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:32.041332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1340/2660] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:32.048816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:32.067105 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc2dbc-2a72-436a-9b9a-c7276609df12 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:32.076822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:32.078168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.078672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.079170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.080667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:32.081319 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:32.084537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:32.110926 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1321/2661] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:34:32.115856 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:32.135218 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa182c16-96b5-433c-aaed-c4497e680672 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:32.142717 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:32.143725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.144199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.144656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.145687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:32.148665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:32.281474 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1341/2662] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:32 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:32.286481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:32.307746 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13ada9-3e61-4076-982f-5e0c0046e0ce admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:32.408919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:32.409606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.409973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.410542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.411794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:32.414226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:32.554676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1322/2663] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:32 2022] POST /identity/v3/users => generated 286 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:32.559397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:32.577693 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f224b4-a2f4-4675-9b8b-b0acd08be969 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:32.587760 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:32.589450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.589898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.590448 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.591666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:32.595187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:32.637492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1342/2664] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:32.648340 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:32.672809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003719e4-3301-4651-811d-a6c8b6e97a58 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:32.679730 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62db1e-0cd2-4670-95bc-4984dac0ebba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:32.680882 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62db1e-0cd2-4670-95bc-4984dac0ebba admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.681403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62db1e-0cd2-4670-95bc-4984dac0ebba admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.681904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62db1e-0cd2-4670-95bc-4984dac0ebba admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.683286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62db1e-0cd2-4670-95bc-4984dac0ebba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:32.685838 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62db1e-0cd2-4670-95bc-4984dac0ebba admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:32.705419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1323/2665] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:32.707127 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:32.721511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d0a78-db72-4bf2-9066-d45d3062d40d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:32.729940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:32.732169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.732169 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.732736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] PATH_INFO: `/v3/projects/fff077a9afa543b2b239071f9c79ac38/users/078a0f5285624b5f82b80e53a3a68ef9/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.760047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff077a9afa543b2b239071f9c79ac38, user_id=078a0f5285624b5f82b80e53a3a68ef9, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:32.762970 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:32.889571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1343/2666] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:32 2022] PUT /identity/v3/projects/fff077a9afa543b2b239071f9c79ac38/users/078a0f5285624b5f82b80e53a3a68ef9/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:32.895106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680c68f-0610-489a-9c27-1e4c5bbae73b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:32.899372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680c68f-0610-489a-9c27-1e4c5bbae73b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:32.899372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680c68f-0610-489a-9c27-1e4c5bbae73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:32.915902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-a680c68f-0610-489a-9c27-1e4c5bbae73b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:32.917205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a680c68f-0610-489a-9c27-1e4c5bbae73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:32.970392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:33.011501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1324/2667] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:33.024535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:33.033705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:33.137417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fa1a6-6960-4102-8a0b-61b3a6bcc970 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:33.141556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:33.142229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:33.142522 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:33.142794 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:33.154068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:33.155949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:33.172334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1344/2668] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:35.364848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:35.366307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:35.366307 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:35.401455 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] MFA Rules not processed for user `078a0f5285624b5f82b80e53a3a68ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:35.402994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:35.515601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1325/2669] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:35.528414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:35.530940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:35.643605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:35.686037 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970e77f-1d4d-4fc5-bc87-90e1934e2196 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:35.692536 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c4289-722d-400a-8444-42c42b5e1860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:35.693592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c4289-722d-400a-8444-42c42b5e1860 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:35.694106 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c4289-722d-400a-8444-42c42b5e1860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:35.694583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c4289-722d-400a-8444-42c42b5e1860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:35.730339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c4289-722d-400a-8444-42c42b5e1860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:35.733077 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c4289-722d-400a-8444-42c42b5e1860 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:35.754477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1345/2670] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:36.255200 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:36.257394 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:36.285323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb690d-2ac9-452e-a9e8-1aee34b5212c None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:36.313606 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:36.314823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:36.330821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:36.330821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:36.335226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:36.512704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:36.515173 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:36.540493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1326/2671] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:37.491199 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:37.492093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:37.492093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:37.527805 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] MFA Rules not processed for user `79755c3f9eec4c5ab3e98038015bff32`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:37.529429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:37.655253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1346/2672] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:37.661138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:37.664132 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:37.683391 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d82d9-21cc-4d92-a012-87701cc491d0 service cinder] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:37.689559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3c33b-c02b-4940-863b-077516faf60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:37.690781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3c33b-c02b-4940-863b-077516faf60e admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:37.691310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3c33b-c02b-4940-863b-077516faf60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:37.691854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3c33b-c02b-4940-863b-077516faf60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:37.692875 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3c33b-c02b-4940-863b-077516faf60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:37.695580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3c33b-c02b-4940-863b-077516faf60e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:37.846089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1327/2673] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:37 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:37.852998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:37.870712 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99054c51-42fa-4093-a355-6fdf2d4532b7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:37.944945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:37.945546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:37.945819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:37.967809 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] MFA Rules not processed for user `078a0f5285624b5f82b80e53a3a68ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:37.969377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:37.985322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:37.986490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:37.986965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:37.987416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:37.989827 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:37.992773 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:38.035619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:38.059330 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1328/2674] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:34:38.126011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:38.126739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1347/2675] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:37 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:38.145221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382866fd-a886-487d-88ed-8ae7a7518403 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:38.153692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:38.154687 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:38.155158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:38.155628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:38.156948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:38.159972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:38.188011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1329/2676] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:38.193399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:38.211062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd77b42-5583-4446-a2ed-787f7c33aa40 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:38.220545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:38.222535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:38.223109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:38.223715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] PATH_INFO: `/v3/projects/8bfd730457a34f2192b1f1cf0d5a1aed/users/10cc3f8486704b77aeed38494dcf0408/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:38.252587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfd730457a34f2192b1f1cf0d5a1aed, user_id=10cc3f8486704b77aeed38494dcf0408, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:38.259237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:38.393922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1348/2677] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:38 2022] PUT /identity/v3/projects/8bfd730457a34f2192b1f1cf0d5a1aed/users/10cc3f8486704b77aeed38494dcf0408/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:38.400333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:38.418873 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a8d32-b6d7-41d8-b1f1-6051284c2778 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:38.487589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:38.491295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:38.519750 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:38.521011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:38.521472 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:38.522804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:38.526139 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:38.526355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:38.548047 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1330/2678] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:38.554872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:38.567454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:38.574403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8efd-ad54-4c57-999c-46065dda6d8a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:38.581080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:38.582325 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:38.583261 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:38.583473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] PATH_INFO: `/v3/projects/8bfd730457a34f2192b1f1cf0d5a1aed/users/10cc3f8486704b77aeed38494dcf0408/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:38.586720 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfd730457a34f2192b1f1cf0d5a1aed, user_id=10cc3f8486704b77aeed38494dcf0408, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:38.590493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:38.612056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1331/2679] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:38 2022] PUT /identity/v3/projects/8bfd730457a34f2192b1f1cf0d5a1aed/users/10cc3f8486704b77aeed38494dcf0408/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:38.614470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703d59d-0831-45d4-b1fc-f0ce5c189fb8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:38.686330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:38.686916 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:38.687195 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:38.687470 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:38.689797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:38.702356 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:38.704111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53236f-b27a-4715-bb1e-91376355be7f service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:38.716444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1349/2680] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:41.080331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:41.092787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:41.257151 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a0666-0c16-49ed-9c1f-faca0ac2b3dc admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:41.269489 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:41.275535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:41.275895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:41.276428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] PATH_INFO: `/v3/users/23bf4abb51dc47839dfcd10e9522006b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:41.278985 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bf4abb51dc47839dfcd10e9522006b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:41.283454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:41.721870 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bf4abb51dc47839dfcd10e9522006b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:41.722622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bf4abb51dc47839dfcd10e9522006b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:41.739033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bf4abb51dc47839dfcd10e9522006b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:41.750938 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:41.774804 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bf4abb51dc47839dfcd10e9522006b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:41.776673 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bf4abb51dc47839dfcd10e9522006b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:41.926257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1332/2681] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:41 2022] DELETE /identity/v3/users/23bf4abb51dc47839dfcd10e9522006b => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:41.929444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:41.930383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:41.930383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:41.964953 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:41.965467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:42.064354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1350/2682] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:42.078784 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:42.086834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:42.199998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e2da-c085-48a2-a4c3-bdca3799a117 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:42.209787 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:42.210991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:42.211473 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:42.211944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:42.225979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:42.228564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:42.251644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1333/2683] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:43.561847 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:43.565358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:43.601266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13644a-c682-4362-9b7d-41bc4d3adec6 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:43.625477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:43.627018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:43.627817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:43.628431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] PATH_INFO: `/v3/projects/5bee02e02b5442d7a801109ceeb349d3` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:43.630790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bee02e02b5442d7a801109ceeb349d3)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:43.639562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:43.943215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bee02e02b5442d7a801109ceeb349d3', 'request_id': 'req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:43.971115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bee02e02b5442d7a801109ceeb349d3', 'request_id': 'req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.213322 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bee02e02b5442d7a801109ceeb349d3', 'request_id': 'req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.255606 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] The token cache is being invalidate because project 5bee02e02b5442d7a801109ceeb349d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:44.258002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.260057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.262186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1351/2684] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:43 2022] DELETE /identity/v3/projects/5bee02e02b5442d7a801109ceeb349d3 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:44.266849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:44.295559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1afac-64f1-4b0f-ace0-205f30275cc8 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:44.509386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:44.510311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:44.510599 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:44.510887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] PATH_INFO: `/v3/users/f6d8033fbada4360a51833819263a55b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:44.513035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d8033fbada4360a51833819263a55b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:44.515080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:44.873434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d8033fbada4360a51833819263a55b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.887320 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d8033fbada4360a51833819263a55b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.919440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d8033fbada4360a51833819263a55b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.953269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d8033fbada4360a51833819263a55b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:44.954959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-50ded839-5610-4aaa-b30e-ca7b76685509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d8033fbada4360a51833819263a55b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:45.131691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:45.181971 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1334/2685] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:34:44 2022] DELETE /identity/v3/users/f6d8033fbada4360a51833819263a55b => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:34:46.086094 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:46.089739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:46.134330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b7f2f-7f8e-44e0-ad89-72641f04a6ec admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:46.158676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:46.253942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:46.255749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:46.256310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:46.256869 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] PATH_INFO: `/v3/projects/ba77c2893f45483ca902169147156c6c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:46.259332 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba77c2893f45483ca902169147156c6c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:46.263351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:46.438142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba77c2893f45483ca902169147156c6c', 'request_id': 'req-325c04c1-6be6-4ab6-ac15-f7515d7e7329'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:46.447316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba77c2893f45483ca902169147156c6c', 'request_id': 'req-325c04c1-6be6-4ab6-ac15-f7515d7e7329'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:46.567306 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba77c2893f45483ca902169147156c6c', 'request_id': 'req-325c04c1-6be6-4ab6-ac15-f7515d7e7329'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:46.586570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] The token cache is being invalidate because project ba77c2893f45483ca902169147156c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:34:46.587087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:46.588253 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-325c04c1-6be6-4ab6-ac15-f7515d7e7329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:34:46.596420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1352/2686] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:34:46 2022] DELETE /identity/v3/projects/ba77c2893f45483ca902169147156c6c => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:46.599366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:46.600515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:46.601022 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:46.629680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:46.631544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:46.718693 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1335/2687] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:46.722097 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:46.722627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:46.723430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:46.760607 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:46.763085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:46.781929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1353/2688] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:46.788830 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:46.807829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd7bc8-16c9-4bbb-9a69-99384330b9cf None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:46.816015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:46.824262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:46.824262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:46.824262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:46.824262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:34:46.824262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:46.824262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:46.854027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1336/2689] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:34:46.855556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:46.867449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab4d80-bede-4c32-a036-bc70db58ee40 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:46.873891 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:46.874834 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:46.875266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:46.875678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:46.876630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:46.879083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:47.102951 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1354/2690] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:46 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:47.108066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:47.144772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f2878-3d5c-46eb-99e4-930deed0c290 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:47.268326 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:47.269363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:47.269864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:47.270348 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:47.272033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:47.275433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:47.424158 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1337/2691] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:47 2022] POST /identity/v3/users => generated 288 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:47.425487 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:47.448157 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eec022-0f65-4526-b784-55e1016e0ac2 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:47.465824 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:47.468809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:47.468809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:47.468809 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:47.470285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:47.473543 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:47.501481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1355/2692] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:47.505645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:47.522183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3582a-bc7c-42ef-bb96-6b64c597118d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:47.530890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3010e-c7a7-4a7d-b1a9-2b4fa1501405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:47.532074 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3010e-c7a7-4a7d-b1a9-2b4fa1501405 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:47.532646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3010e-c7a7-4a7d-b1a9-2b4fa1501405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:47.533116 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3010e-c7a7-4a7d-b1a9-2b4fa1501405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:47.534423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3010e-c7a7-4a7d-b1a9-2b4fa1501405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:47.537594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3010e-c7a7-4a7d-b1a9-2b4fa1501405 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:47.552339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1338/2693] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:47.556652 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:47.570815 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2412-3243-4517-9e08-48ab87b7236f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:47.576813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:47.577822 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:47.578206 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:47.578558 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] PATH_INFO: `/v3/projects/b8349eff076f4e53810c9f77498af0c7/users/28268844ae75482b9ededd8f530fe7ef/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:47.601330 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8349eff076f4e53810c9f77498af0c7, user_id=28268844ae75482b9ededd8f530fe7ef, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:47.604358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:47.688504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1356/2694] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:47 2022] PUT /identity/v3/projects/b8349eff076f4e53810c9f77498af0c7/users/28268844ae75482b9ededd8f530fe7ef/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:47.701372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:47.702250 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:47.702412 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:47.760604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] MFA Rules not processed for user `28268844ae75482b9ededd8f530fe7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:47.762850 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:47.920019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:47.943775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1339/2695] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:47.949792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:48.067000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:48.093182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885dbe-459b-4d39-a5e4-e7c363ea282b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:48.101819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:48.103118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:48.104754 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:48.105670 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:48.106978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:48.109843 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:48.269597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:48.294966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1357/2696] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:47 2022] POST /identity/v3/projects => generated 373 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:48.317920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb24d8-0de5-4b0d-a2c1-c815f5b644f2 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:48.450216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:48.450216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:48.450216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:48.450216 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:48.450979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:48.450979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:48.571005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1340/2697] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:48 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:48.577019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:48.597978 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc6bf-dca2-47ae-a684-1cbfb9e7c3d8 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:48.629663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:48.630123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:48.630421 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:48.630737 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:48.631583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:48.633480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:48.652688 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1358/2698] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:48.660831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:48.680890 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525670e-1449-4bd2-a6e6-ad4852b9b4f3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:48.687631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4178b7-ec2f-4613-8bbf-5a8f72afb74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:48.690620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4178b7-ec2f-4613-8bbf-5a8f72afb74b admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:48.692364 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4178b7-ec2f-4613-8bbf-5a8f72afb74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:48.693283 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4178b7-ec2f-4613-8bbf-5a8f72afb74b admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:48.696405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4178b7-ec2f-4613-8bbf-5a8f72afb74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:48.701178 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4178b7-ec2f-4613-8bbf-5a8f72afb74b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:48.725166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1341/2699] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:48.730928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:48.743591 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf278-8c21-4483-a344-870c61f0c952 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:48.751456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:48.752420 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:48.752702 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:48.752991 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] PATH_INFO: `/v3/projects/22d4d29ae2fd41f78b6581ce544181c3/users/be28e4bb876f440f8c2ce9c017c3a68d/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:48.774359 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d4d29ae2fd41f78b6581ce544181c3, user_id=be28e4bb876f440f8c2ce9c017c3a68d, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:48.777304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:48.903748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1359/2700] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:48 2022] PUT /identity/v3/projects/22d4d29ae2fd41f78b6581ce544181c3/users/be28e4bb876f440f8c2ce9c017c3a68d/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:48.924095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc55869-f737-4c2e-b8bc-318dd112a4e3 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:48.924095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc55869-f737-4c2e-b8bc-318dd112a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:48.924095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc55869-f737-4c2e-b8bc-318dd112a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:48.962779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-0fc55869-f737-4c2e-b8bc-318dd112a4e3 None None] MFA Rules not processed for user `be28e4bb876f440f8c2ce9c017c3a68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:48.964048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0fc55869-f737-4c2e-b8bc-318dd112a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:49.128649 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1342/2701] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:49.143695 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:49.277433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052336-fe61-464b-ac09-f7348a554e19 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:49.292780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:49.294088 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:49.294941 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:49.295472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:49.297462 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:49.302230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:49.325293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:49.325625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:49.325861 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:49.394765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] MFA Rules not processed for user `10cc3f8486704b77aeed38494dcf0408`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:49.397226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:49.487656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1360/2702] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:34:49 2022] POST /identity/v3/projects => generated 377 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:34:49.494309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:49.513879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:49.546699 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1343/2703] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:49.569124 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:49.570913 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:49.641625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31651-181a-4723-9807-170bf251e9b1 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:49.657101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:49.657797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:49.658089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:49.658373 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:49.660011 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:49.662689 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:49.788803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df179bd-b032-45bc-8838-3db07c29e830 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:49.856495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1361/2704] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:34:49 2022] POST /identity/v3/users => generated 295 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:34:49.871910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:49.886703 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d3827-d8e9-4b60-881a-1c89ade95832 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:49.892339 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:49.893042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:49.893336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:49.893611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:49.894561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:49.896237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:49.898720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:49.915155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1362/2705] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:34:49.921370 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:49.923799 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:49.924561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:49.924848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:49.925175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:49.927480 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:49.934287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f822219-472d-4837-9de8-7d781d453cd9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:49.940663 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:49.942329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:49.942329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:49.942329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] PATH_INFO: `/v3/projects/5244463fc4ee4bc5829141c12f1cb24f/users/8d4511be75c649eea369e96e7c6c9375/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:49.964429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5244463fc4ee4bc5829141c12f1cb24f, user_id=8d4511be75c649eea369e96e7c6c9375, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:49.967895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:50.016625 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:50.020027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:50.046634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1344/2706] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:50.067301 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1363/2707] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:49 2022] PUT /identity/v3/projects/5244463fc4ee4bc5829141c12f1cb24f/users/8d4511be75c649eea369e96e7c6c9375/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:50.079021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:50.102199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35964d-8436-44d5-a0e3-22b32c5caaf6 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:50.185086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090730d-6542-43f4-9cd6-6cfa6ca31171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:50.185966 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090730d-6542-43f4-9cd6-6cfa6ca31171 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:50.186182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090730d-6542-43f4-9cd6-6cfa6ca31171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:50.186333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090730d-6542-43f4-9cd6-6cfa6ca31171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:50.187048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090730d-6542-43f4-9cd6-6cfa6ca31171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:50.188847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090730d-6542-43f4-9cd6-6cfa6ca31171 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:50.215096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1345/2708] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:34:50.215581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:50.233605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4decf4e-6571-4985-b8d8-dc09e6abac58 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:50.240926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:50.241858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:50.242140 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:50.242419 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] PATH_INFO: `/v3/projects/5244463fc4ee4bc5829141c12f1cb24f/users/8d4511be75c649eea369e96e7c6c9375/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:50.245400 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5244463fc4ee4bc5829141c12f1cb24f, user_id=8d4511be75c649eea369e96e7c6c9375, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:50.248437 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:50.272329 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1364/2709] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:34:50 2022] PUT /identity/v3/projects/5244463fc4ee4bc5829141c12f1cb24f/users/8d4511be75c649eea369e96e7c6c9375/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:34:50.373484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8f49-38b6-4f57-8347-d743ea6f726f None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:50.373855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8f49-38b6-4f57-8347-d743ea6f726f None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:50.374059 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8f49-38b6-4f57-8347-d743ea6f726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:50.388138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:34:50.420278 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8e4c8f49-38b6-4f57-8347-d743ea6f726f None None] MFA Rules not processed for user `8d4511be75c649eea369e96e7c6c9375`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:50.430292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8e4c8f49-38b6-4f57-8347-d743ea6f726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:50.560334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1346/2710] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:50.572593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:50.574442 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:50.656025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ad094-29b6-4959-8668-34f38a72fd90 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:50.661238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:50.662642 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:50.663056 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:50.663398 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:50.762348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:50.765385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:50.784486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1365/2711] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:34:54.648228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:54.649054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:54.649054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:54.669438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] MFA Rules not processed for user `79755c3f9eec4c5ab3e98038015bff32`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:34:54.670725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:54.739409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1347/2712] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:34:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:34:56.804731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:56.806789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:34:56.830495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16065ffb-0729-4375-aa6e-62bf44dc8338 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:56.835490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:56.836252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:56.836614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:56.836907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:56.918526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:56.921823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:56.940196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1366/2713] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:34:59.214101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:34:59.227535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d80cd-ed52-45db-addf-8078b5fdf192 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:34:59.232859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d429-18b8-400e-9bd8-d305266c59ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:34:59.233884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d429-18b8-400e-9bd8-d305266c59ce service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:34:59.234004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d429-18b8-400e-9bd8-d305266c59ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:34:59.234695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d429-18b8-400e-9bd8-d305266c59ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:34:59.246629 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2d429-18b8-400e-9bd8-d305266c59ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:34:59.248113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2d429-18b8-400e-9bd8-d305266c59ce service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:34:59.259925 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1348/2714] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:34:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:35:06.939336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=93,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:06.951120 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:07.065675 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718d985-409a-4edc-88e4-6974d3c76be8 service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:07.072188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:07.073189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:07.073479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:07.073751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] PATH_INFO: `/v3/users/28268844ae75482b9ededd8f530fe7ef` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:07.075076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28268844ae75482b9ededd8f530fe7ef)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:07.077162 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:07.378743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28268844ae75482b9ededd8f530fe7ef'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:07.404104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28268844ae75482b9ededd8f530fe7ef'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:07.405540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28268844ae75482b9ededd8f530fe7ef'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:07.432371 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28268844ae75482b9ededd8f530fe7ef'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:07.433518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28268844ae75482b9ededd8f530fe7ef'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:07.584524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1367/2715] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:06 2022] DELETE /identity/v3/users/28268844ae75482b9ededd8f530fe7ef => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:07.589822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:07.590698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:07.590932 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:07.612998 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:07.614457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:07.666454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:07.698310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1349/2716] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:07.713771 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:07.716491 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:07.834444 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6e028-155e-48b0-a996-e4d8107d30c2 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:07.842149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:07.843507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:07.843988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:07.844541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:07.863208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:07.866128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:07.880649 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1368/2717] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:08.156664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:08.159150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:08.187206 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20ec6f-70bc-4642-9ff3-2d42ba4df37d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:08.201248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:08.201248 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:08.201901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:08.202285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] PATH_INFO: `/v3/users/a7e576a30ce441c38fe78b41a8b90b9c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:08.204500 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e576a30ce441c38fe78b41a8b90b9c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:08.206885 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:08.527810 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e576a30ce441c38fe78b41a8b90b9c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.529048 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e576a30ce441c38fe78b41a8b90b9c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.540503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e576a30ce441c38fe78b41a8b90b9c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.556915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e576a30ce441c38fe78b41a8b90b9c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.569293 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-b93c24a8-3768-4271-8244-60650f450529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e576a30ce441c38fe78b41a8b90b9c'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.641363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:08.648897 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:08.659350 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88caa2-8559-49b6-8164-d86a9e77c9ee service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:08.665217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:08.666147 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:08.666436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:08.666707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] PATH_INFO: `/v3/projects/b8349eff076f4e53810c9f77498af0c7` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:08.668138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8349eff076f4e53810c9f77498af0c7)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:08.670155 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:08.702181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:08.752709 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1350/2718] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:08 2022] DELETE /identity/v3/users/a7e576a30ce441c38fe78b41a8b90b9c => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:08.754977 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:08.755372 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:08.755656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:08.775413 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:08.776742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:08.852076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8349eff076f4e53810c9f77498af0c7', 'request_id': 'req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.857662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1351/2719] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:08.870324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8349eff076f4e53810c9f77498af0c7', 'request_id': 'req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:08.870650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:08.872571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:08.973363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ab86c-a01f-40fd-8de7-a6b8b12984e1 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:08.980485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:08.992490 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8349eff076f4e53810c9f77498af0c7', 'request_id': 'req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.006709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] The token cache is being invalidate because project b8349eff076f4e53810c9f77498af0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:09.007408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.008310 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.010743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1369/2720] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:08 2022] DELETE /identity/v3/projects/b8349eff076f4e53810c9f77498af0c7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:09.017321 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:09.023047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:09.053170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:09.054309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:09.054708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:09.055110 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:09.059438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:09.131523 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fff72a-54e6-43f4-92d2-4c9e0ba11360 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:09.140366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:09.141998 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:09.142494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:09.142995 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] PATH_INFO: `/v3/users/be28e4bb876f440f8c2ce9c017c3a68d` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:09.145430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be28e4bb876f440f8c2ce9c017c3a68d)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:09.149512 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:09.155646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:09.157221 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:09.169280 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1352/2721] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:35:09.620740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be28e4bb876f440f8c2ce9c017c3a68d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.635966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be28e4bb876f440f8c2ce9c017c3a68d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.638221 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be28e4bb876f440f8c2ce9c017c3a68d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.677782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be28e4bb876f440f8c2ce9c017c3a68d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.679292 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be28e4bb876f440f8c2ce9c017c3a68d'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:09.827088 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:09.829635 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:09.836493 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1370/2722] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:09 2022] DELETE /identity/v3/users/be28e4bb876f440f8c2ce9c017c3a68d => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:09.858117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27b1f1-0bad-488e-b300-9f790518e7e6 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:09.958953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:09.959952 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:09.960440 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:09.960723 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] PATH_INFO: `/v3/projects/08358393d5c5450089399204a38a5035` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:09.962504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=08358393d5c5450089399204a38a5035)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:09.964975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:10.186284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08358393d5c5450089399204a38a5035', 'request_id': 'req-46a662bc-f512-438a-96c2-42d031ff2afd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.193783 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08358393d5c5450089399204a38a5035', 'request_id': 'req-46a662bc-f512-438a-96c2-42d031ff2afd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.198087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:10.309964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08358393d5c5450089399204a38a5035', 'request_id': 'req-46a662bc-f512-438a-96c2-42d031ff2afd'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.330569 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] The token cache is being invalidate because project 08358393d5c5450089399204a38a5035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:10.331077 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.332162 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-46a662bc-f512-438a-96c2-42d031ff2afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.335855 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1353/2723] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:09 2022] DELETE /identity/v3/projects/08358393d5c5450089399204a38a5035 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:10.341524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:10.343845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:10.363104 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8170d1-2d0b-4d5e-bc40-5e754279f858 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:10.411229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:10.427860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:10.428748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:10.429603 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:10.471131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] MFA Rules not processed for user `078a0f5285624b5f82b80e53a3a68ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:10.473620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:10.492751 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:10.494264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:10.494589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:10.494872 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] PATH_INFO: `/v3/users/c688d41577204551bd046775bc864d50` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:10.496686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c688d41577204551bd046775bc864d50)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:10.499057 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:10.578315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1354/2724] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:10.579557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:10.581831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:10.599084 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad6ff3-0a75-4aa1-91ee-9983ee9c44ad None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:10.605118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:10.606143 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:10.606541 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:10.606921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] PATH_INFO: `/v3/projects/22d4d29ae2fd41f78b6581ce544181c3` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:10.608515 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d4d29ae2fd41f78b6581ce544181c3)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:10.611041 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:10.939046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c688d41577204551bd046775bc864d50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.944476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d4d29ae2fd41f78b6581ce544181c3', 'request_id': 'req-67798c3d-19b8-48e1-a65f-be955b948a96'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.952246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c688d41577204551bd046775bc864d50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.953256 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c688d41577204551bd046775bc864d50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.957768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d4d29ae2fd41f78b6581ce544181c3', 'request_id': 'req-67798c3d-19b8-48e1-a65f-be955b948a96'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.972524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c688d41577204551bd046775bc864d50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:10.973525 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c688d41577204551bd046775bc864d50'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.068826 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d4d29ae2fd41f78b6581ce544181c3', 'request_id': 'req-67798c3d-19b8-48e1-a65f-be955b948a96'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.083978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] The token cache is being invalidate because project 22d4d29ae2fd41f78b6581ce544181c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:11.084694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.085546 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.087478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1355/2725] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:10 2022] DELETE /identity/v3/projects/22d4d29ae2fd41f78b6581ce544181c3 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:11.093212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:11.094244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:11.161766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1371/2726] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:10 2022] DELETE /identity/v3/users/c688d41577204551bd046775bc864d50 => generated 0 bytes in 821 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:11.203219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67798c3d-19b8-48e1-a65f-be955b948a96 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:11.275525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:11.276481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:11.277536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:11.277536 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] PATH_INFO: `/v3/users/8d4511be75c649eea369e96e7c6c9375` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:11.278398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4511be75c649eea369e96e7c6c9375)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:11.280429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:11.570953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4511be75c649eea369e96e7c6c9375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.572038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4511be75c649eea369e96e7c6c9375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.587829 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4511be75c649eea369e96e7c6c9375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.615019 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4511be75c649eea369e96e7c6c9375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.616256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-3b856fba-0c86-4ea4-a54f-1dd683aca8fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4511be75c649eea369e96e7c6c9375'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:11.647392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:11.808390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1356/2727] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:11 2022] DELETE /identity/v3/users/8d4511be75c649eea369e96e7c6c9375 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:11.814790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:11.817052 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:11.841706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5120-d22d-4878-97dc-53c790cb0431 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:11.956601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:11.958651 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:11.959047 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:11.959450 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] PATH_INFO: `/v3/projects/4689727bce874b83a02ed9e7f34e417b` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:11.961275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4689727bce874b83a02ed9e7f34e417b)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:11.963599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:12.185885 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4689727bce874b83a02ed9e7f34e417b', 'request_id': 'req-f94992b3-4410-4f37-b033-61a69921c296'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.202026 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4689727bce874b83a02ed9e7f34e417b', 'request_id': 'req-f94992b3-4410-4f37-b033-61a69921c296'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.235055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:12.325774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4689727bce874b83a02ed9e7f34e417b', 'request_id': 'req-f94992b3-4410-4f37-b033-61a69921c296'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.339445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] The token cache is being invalidate because project 4689727bce874b83a02ed9e7f34e417b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:12.342862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.342862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.342862 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1372/2728] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:11 2022] DELETE /identity/v3/projects/4689727bce874b83a02ed9e7f34e417b => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:12.348509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c0c4c-82e6-453e-94e1-3a145e70b54b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.349194 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c0c4c-82e6-453e-94e1-3a145e70b54b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.349588 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c0c4c-82e6-453e-94e1-3a145e70b54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.380615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:12.380615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:12.385435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-bd0c0c4c-82e6-453e-94e1-3a145e70b54b None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:12.395762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-bd0c0c4c-82e6-453e-94e1-3a145e70b54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:12.474628 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94992b3-4410-4f37-b033-61a69921c296 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:12.480434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:12.481348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.481665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.481937 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] PATH_INFO: `/v3/projects/5244463fc4ee4bc5829141c12f1cb24f` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.483509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5244463fc4ee4bc5829141c12f1cb24f)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:12.485570 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:12.511895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1357/2729] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:12.520851 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.521353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.521860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.556574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:12.558496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:12.572410 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1358/2730] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:12.577011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:12.587341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84a933-c992-47ff-9a56-72bb52fd2931 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:12.593379 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:12.594315 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.594727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.595104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.596055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:35:12.596563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:12.598433 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:12.614192 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1359/2731] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:35:12.621332 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:12.633251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ead6ee-8dd4-4653-898c-b0c6bc700e1f admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:12.638565 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:12.639308 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.639637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.639958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.640827 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:12.642860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:12.683672 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5244463fc4ee4bc5829141c12f1cb24f', 'request_id': 'req-ba73325b-9a6f-4f2e-9985-03f5c01152f9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.690728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5244463fc4ee4bc5829141c12f1cb24f', 'request_id': 'req-ba73325b-9a6f-4f2e-9985-03f5c01152f9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.821831 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5244463fc4ee4bc5829141c12f1cb24f', 'request_id': 'req-ba73325b-9a6f-4f2e-9985-03f5c01152f9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.834083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1360/2732] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:35:12 2022] POST /identity/v3/projects => generated 367 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:12.842974 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:12.852898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] The token cache is being invalidate because project 5244463fc4ee4bc5829141c12f1cb24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:12.853376 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.856251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ba73325b-9a6f-4f2e-9985-03f5c01152f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:12.858974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1373/2733] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:12 2022] DELETE /identity/v3/projects/5244463fc4ee4bc5829141c12f1cb24f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:12.868798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7145125-e513-4a97-a98e-0ca743d357d9 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.872480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7145125-e513-4a97-a98e-0ca743d357d9 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.872480 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7145125-e513-4a97-a98e-0ca743d357d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.900252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-f7145125-e513-4a97-a98e-0ca743d357d9 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:12.902207 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f7145125-e513-4a97-a98e-0ca743d357d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:12.951167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e995f-5fac-4c5b-9b65-e6c2ce24001e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:12.959070 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:12.960133 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:12.960637 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:12.961092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:12.962300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:12.965454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:12.994115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1374/2734] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:13.003415 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.003848 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.004265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.024413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:13.025657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:13.038729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1375/2735] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:13.042021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.051975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccac8e-6f27-4fc3-a4fb-71582f046801 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.057280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.057960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.058242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.058517 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.059258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:35:13.059563 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.061238 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.074367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1376/2736] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:35:13.078867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.090164 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107246-9b43-4596-93a3-4c66f51085c9 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.095506 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.096228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.096624 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.096945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.097705 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.100252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.131363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1361/2737] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:35:12 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:13.137645 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.139448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:13.152409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9c703-645b-4fa8-b5b6-d409500bd047 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.160582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.162358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.162848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.163351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.165008 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.168185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.185955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1362/2738] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:13.190795 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.205916 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45856d81-1d6c-4e25-aca5-f540a3ea2bd2 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.215686 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1377/2739] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:13.215986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.215986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.215986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.215986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.216607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.217541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.219268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.236439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1363/2740] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:13.243358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.267951 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302a226-9e84-4aca-8635-7936f7ff4835 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.328212 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef293d4-138e-419c-bb7e-b4b286509d48 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.337449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.338656 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.339172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.339668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.341149 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.342948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:13.345240 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.368147 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00b927-ee40-453f-953c-2116f56e947b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.369113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00b927-ee40-453f-953c-2116f56e947b admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.369401 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00b927-ee40-453f-953c-2116f56e947b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.369685 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00b927-ee40-453f-953c-2116f56e947b admin admin] PATH_INFO: `/v3/projects/b58e501fc8e64ea5853b61dd864a1895/users/35b76e7cddf94d8495e08e625d9d2267/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.387052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00b927-ee40-453f-953c-2116f56e947b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58e501fc8e64ea5853b61dd864a1895, user_id=35b76e7cddf94d8495e08e625d9d2267, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.389801 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00b927-ee40-453f-953c-2116f56e947b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.488167 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1364/2741] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:13 2022] PUT /identity/v3/projects/b58e501fc8e64ea5853b61dd864a1895/users/35b76e7cddf94d8495e08e625d9d2267/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:13.510029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.510503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.511044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.557934 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] MFA Rules not processed for user `35b76e7cddf94d8495e08e625d9d2267`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:13.559295 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:13.612729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:13.615832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1378/2742] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/users => generated 288 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:13.622392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.642152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b589a-5628-4a6c-b4a8-85df6c8e227d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.684697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1365/2743] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:13.690492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.746073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.746865 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.747245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.747678 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.748723 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.750745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.765107 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1379/2744] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:13.770386 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.783564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae4d31-f7aa-4970-b715-32ef2c6aeb36 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.784157 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cce03-0fc6-47a0-9ca6-da29cdb1c9fc None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.790436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.791063 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.791402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.791798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.792138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.792441 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.792713 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.793094 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.793575 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.794471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.795534 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.796802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.808478 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1380/2745] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:13.814138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.827382 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9ef2-a4b3-47d8-8eb2-d183e73ba64c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:13.832518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3a532-698f-4427-a3c3-34c0b2fe9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:13.833985 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a532-698f-4427-a3c3-34c0b2fe9833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.834270 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a532-698f-4427-a3c3-34c0b2fe9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.834559 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a532-698f-4427-a3c3-34c0b2fe9833 admin admin] PATH_INFO: `/v3/projects/ebe772f995f743118963aa8c85ab5c2b/users/8f357411d6a547468e94697978f9ca6f/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:13.854237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3a532-698f-4427-a3c3-34c0b2fe9833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe772f995f743118963aa8c85ab5c2b, user_id=8f357411d6a547468e94697978f9ca6f, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:13.857078 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3a532-698f-4427-a3c3-34c0b2fe9833 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:13.935175 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1366/2746] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/projects => generated 367 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:13.939719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:13.968275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1381/2747] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:13 2022] PUT /identity/v3/projects/ebe772f995f743118963aa8c85ab5c2b/users/8f357411d6a547468e94697978f9ca6f/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:13.993109 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:13.993562 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:13.994006 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.039040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] MFA Rules not processed for user `8f357411d6a547468e94697978f9ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:14.041209 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:14.043679 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51fcb-bced-4450-a16d-5a9e027bad86 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:14.119765 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:14.120504 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.120698 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.120978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.121695 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:14.123749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:14.154622 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1382/2748] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:14.166552 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:14.168245 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:14.232841 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1367/2749] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:35:13 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:14.235475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:14.256487 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab34a78-1522-4c47-809b-76b733f0674c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:14.266284 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:14.268450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.269148 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.273496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.280817 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139d47d-0076-4fa4-b6c5-cbe9bc23781c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:14.282044 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:14.286245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:14.289014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:14.292378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.294433 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.295197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.307153 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:14.323232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:14.326705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1368/2750] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:35:14.327334 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:14.333038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:14.344516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1383/2751] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:35:14.360160 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817fcb9-17ad-4beb-9497-1689df585ac6 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:14.370290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:14.373494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.374378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.375091 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] PATH_INFO: `/v3/projects/dbd1392441214f8bbb604a6bcd4abe71/users/16ace7d9bffb43b4b76503a4111f6d57/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.407009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd1392441214f8bbb604a6bcd4abe71, user_id=16ace7d9bffb43b4b76503a4111f6d57, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:14.413149 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:14.504318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1369/2752] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:14 2022] PUT /identity/v3/projects/dbd1392441214f8bbb604a6bcd4abe71/users/16ace7d9bffb43b4b76503a4111f6d57/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:14.511363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:14.540768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:14.544828 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ae99-6b98-4c54-aa96-443d5be3897f service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:14.653105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe292e-0feb-45ac-9fd6-969c240f16cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:14.654265 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe292e-0feb-45ac-9fd6-969c240f16cf admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.654774 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe292e-0feb-45ac-9fd6-969c240f16cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.655286 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe292e-0feb-45ac-9fd6-969c240f16cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.656842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe292e-0feb-45ac-9fd6-969c240f16cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:14.660035 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe292e-0feb-45ac-9fd6-969c240f16cf admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:14.678726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1384/2753] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:35:14.685717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:14.705128 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5076fb-d9b6-4f3f-bd2e-ef38bcfa6d45 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:14.713911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:14.715257 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.715751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.716235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] PATH_INFO: `/v3/projects/dbd1392441214f8bbb604a6bcd4abe71/users/16ace7d9bffb43b4b76503a4111f6d57/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.721191 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd1392441214f8bbb604a6bcd4abe71, user_id=16ace7d9bffb43b4b76503a4111f6d57, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:14.730402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:14.764597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1370/2754] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:14 2022] PUT /identity/v3/projects/dbd1392441214f8bbb604a6bcd4abe71/users/16ace7d9bffb43b4b76503a4111f6d57/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:14.782358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:14.782743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:14.783028 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:14.823405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] MFA Rules not processed for user `16ace7d9bffb43b4b76503a4111f6d57`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:14.826200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:14.832915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:14.955086 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:14.990744 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1385/2755] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:15.000238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:15.003006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:15.101131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba680021-b51f-4915-93cd-b8a114eb7c70 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:15.105295 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08cabe-61c2-4c40-978e-31ae160e4380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:15.105990 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08cabe-61c2-4c40-978e-31ae160e4380 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:15.106254 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08cabe-61c2-4c40-978e-31ae160e4380 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:15.106535 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08cabe-61c2-4c40-978e-31ae160e4380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:15.117955 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08cabe-61c2-4c40-978e-31ae160e4380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:15.119458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08cabe-61c2-4c40-978e-31ae160e4380 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:15.131678 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1371/2756] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:15.470721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:15.489986 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e2838-20ae-4493-aa59-7bea248827b7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:15.497504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:15.498714 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:15.500229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:15.500229 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:15.584721 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:15.586495 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:15.604555 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1386/2757] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:16.401353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcb301-18bd-4af7-977a-4854695f84d8 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:16.401919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcb301-18bd-4af7-977a-4854695f84d8 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:16.402219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcb301-18bd-4af7-977a-4854695f84d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:16.421694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-c9fcb301-18bd-4af7-977a-4854695f84d8 None None] MFA Rules not processed for user `8f357411d6a547468e94697978f9ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:16.424099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-c9fcb301-18bd-4af7-977a-4854695f84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:16.524945 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1372/2758] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:23.691128 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:23.899431 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33e5f-6702-437b-8383-5942b971655c service cinder] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:23.909251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:23.911434 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:23.912025 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:23.912668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] PATH_INFO: `/v3/users/35b76e7cddf94d8495e08e625d9d2267` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:23.915123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b76e7cddf94d8495e08e625d9d2267)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:23.919146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:24.193000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b76e7cddf94d8495e08e625d9d2267'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:24.217388 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b76e7cddf94d8495e08e625d9d2267'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:24.218482 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b76e7cddf94d8495e08e625d9d2267'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:24.242908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:24.251882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b76e7cddf94d8495e08e625d9d2267'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:24.253133 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b76e7cddf94d8495e08e625d9d2267'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:24.358820 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1387/2759] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:23 2022] DELETE /identity/v3/users/35b76e7cddf94d8495e08e625d9d2267 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:24.367025 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:24.367618 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:24.368138 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:24.398042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:24.402814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:24.494329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1373/2760] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:24.508706 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:24.510242 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:24.622345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955336a-a0c6-40e9-97f1-57a8937370ad admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:24.628725 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:24.630408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:24.630910 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:24.631403 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:24.649876 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:24.652781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:24.673367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1388/2761] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:25.518531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:25.520114 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:25.536341 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:25.546442 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773d338-90fd-4623-ace2-a82673cc6ccd None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:25.563065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:25.563065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:25.563065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:25.563065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] PATH_INFO: `/v3/projects/b58e501fc8e64ea5853b61dd864a1895` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:25.563849 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58e501fc8e64ea5853b61dd864a1895)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:25.564878 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:25.795289 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58e501fc8e64ea5853b61dd864a1895', 'request_id': 'req-0101dd3c-6534-4f87-8597-adbf8a56a2f7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:25.810005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58e501fc8e64ea5853b61dd864a1895', 'request_id': 'req-0101dd3c-6534-4f87-8597-adbf8a56a2f7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:25.914805 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58e501fc8e64ea5853b61dd864a1895', 'request_id': 'req-0101dd3c-6534-4f87-8597-adbf8a56a2f7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:25.928997 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] The token cache is being invalidate because project b58e501fc8e64ea5853b61dd864a1895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:25.929392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:25.930209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:25.932388 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1374/2762] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:25 2022] DELETE /identity/v3/projects/b58e501fc8e64ea5853b61dd864a1895 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:25.936849 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:25.948627 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb852a6-0a01-4fe4-bac6-c2e98dc322d0 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:26.050830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:26.052317 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:26.052620 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:26.052898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] PATH_INFO: `/v3/users/16ace7d9bffb43b4b76503a4111f6d57` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:26.054189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ace7d9bffb43b4b76503a4111f6d57)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:26.056175 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:26.336328 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ace7d9bffb43b4b76503a4111f6d57'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:26.346573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ace7d9bffb43b4b76503a4111f6d57'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:26.347428 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ace7d9bffb43b4b76503a4111f6d57'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:26.373832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ace7d9bffb43b4b76503a4111f6d57'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:26.374718 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-5c316c86-70ac-40be-86b3-8e1f889cf9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ace7d9bffb43b4b76503a4111f6d57'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:26.473740 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1389/2763] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:25 2022] DELETE /identity/v3/users/16ace7d9bffb43b4b76503a4111f6d57 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:26.994281 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:26.996530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:27.025904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101dd3c-6534-4f87-8597-adbf8a56a2f7 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:27.145574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:27.147083 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:27.147574 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:27.148055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] PATH_INFO: `/v3/projects/dbd1392441214f8bbb604a6bcd4abe71` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:27.150725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd1392441214f8bbb604a6bcd4abe71)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:27.154453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:27.310176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd1392441214f8bbb604a6bcd4abe71', 'request_id': 'req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:27.318737 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd1392441214f8bbb604a6bcd4abe71', 'request_id': 'req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:27.418154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd1392441214f8bbb604a6bcd4abe71', 'request_id': 'req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:27.438491 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] The token cache is being invalidate because project dbd1392441214f8bbb604a6bcd4abe71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:27.439154 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:27.440209 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-7b167dd9-1d69-4ff6-b6de-11ca11dd0832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:27.444163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1375/2764] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:26 2022] DELETE /identity/v3/projects/dbd1392441214f8bbb604a6bcd4abe71 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:27.452008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:27.452874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:27.453055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:27.484900 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:27.496053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:27.504402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:27.607320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1390/2765] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:27.627418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:27.627786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:27.628131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:27.652929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:27.655127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:27.668794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1376/2766] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:27.680907 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:27.695615 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb20431-52b6-4ace-9ef0-fa9dd034148a None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:27.709153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:27.709153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:27.709153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:27.709153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:27.709842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:35:27.709842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:27.709842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:27.734465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1391/2767] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:35:27.742012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:27.761249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8befc2-15d4-4cd8-8f16-06d6cd7bc07a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:27.772219 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:27.773727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:27.774239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:27.774763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:27.776123 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:27.780350 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:27.789542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:27.933612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1377/2768] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:35:27 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:35:27.939734 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:27.954977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf28c-9301-4222-8ce5-6b51265d1c0b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:28.036918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:28.037826 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:28.038111 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:28.038402 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:28.039126 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:28.041131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:28.148906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1392/2769] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:35:27 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:35:28.155172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:28.173979 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ad7d0-427c-4221-8a39-76868864f46a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:28.178230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:28.179050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:28.179436 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:28.179812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:28.180674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:28.183106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:28.206692 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1378/2770] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:28.212467 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:28.237784 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70efc9-4f99-4c51-8676-7dff9d640d5b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:28.246399 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3ecd9-39b6-4d54-90bd-1ed5669b29c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:28.251680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3ecd9-39b6-4d54-90bd-1ed5669b29c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:28.251680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3ecd9-39b6-4d54-90bd-1ed5669b29c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:28.251680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3ecd9-39b6-4d54-90bd-1ed5669b29c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:28.251680 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3ecd9-39b6-4d54-90bd-1ed5669b29c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:28.253013 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3ecd9-39b6-4d54-90bd-1ed5669b29c3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:28.269346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1393/2771] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:28.276984 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:28.295513 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d74b88-81b2-41f6-93c3-3591d426a327 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:28.303230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:28.304465 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:28.304929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:28.305389 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] PATH_INFO: `/v3/projects/624a87f068e74e94898c36b82fd19f6b/users/1e623f3f04b744bcacce2be8c00d02ca/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:28.326758 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624a87f068e74e94898c36b82fd19f6b, user_id=1e623f3f04b744bcacce2be8c00d02ca, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:28.329991 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:28.537286 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1379/2772] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:28 2022] PUT /identity/v3/projects/624a87f068e74e94898c36b82fd19f6b/users/1e623f3f04b744bcacce2be8c00d02ca/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:28.557445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:28.558378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:28.558378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:28.603699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] MFA Rules not processed for user `1e623f3f04b744bcacce2be8c00d02ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:28.605654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:28.750668 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1394/2773] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:28.765887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:28.768908 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:28.899884 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44fa02-6859-48b4-b7e8-d76db796b272 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:28.905725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0c72f-7234-408c-8ed3-bdaee58da26a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:28.906561 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c72f-7234-408c-8ed3-bdaee58da26a service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:28.906927 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c72f-7234-408c-8ed3-bdaee58da26a service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:28.907291 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c72f-7234-408c-8ed3-bdaee58da26a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:28.922052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0c72f-7234-408c-8ed3-bdaee58da26a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:28.924597 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0c72f-7234-408c-8ed3-bdaee58da26a service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:28.937537 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1380/2774] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:28.968039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:28.976741 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:29.090254 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81503d9-cf5d-4943-815a-ddeb242bedc0 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:29.102770 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:29.104952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:29.105262 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:29.105874 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] PATH_INFO: `/v3/users/1e623f3f04b744bcacce2be8c00d02ca` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:29.108185 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e623f3f04b744bcacce2be8c00d02ca)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:29.112153 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:29.389764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e623f3f04b744bcacce2be8c00d02ca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:29.402860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e623f3f04b744bcacce2be8c00d02ca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:29.403870 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e623f3f04b744bcacce2be8c00d02ca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:29.419348 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e623f3f04b744bcacce2be8c00d02ca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:29.420707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e623f3f04b744bcacce2be8c00d02ca'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:29.555633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1395/2775] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:28 2022] DELETE /identity/v3/users/1e623f3f04b744bcacce2be8c00d02ca => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:29.561930 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:29.563775 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:29.564328 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:29.596906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:29.599378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:29.698573 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1381/2776] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:29.705926 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:29.708076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:29.779855 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350f57e-14e4-415b-b003-aa2a1f7a5b8d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:29.784019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4fa28-f262-44d7-9dd8-6a410dd7c51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:29.784733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4fa28-f262-44d7-9dd8-6a410dd7c51b service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:29.785019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4fa28-f262-44d7-9dd8-6a410dd7c51b service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:29.785289 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4fa28-f262-44d7-9dd8-6a410dd7c51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:29.796569 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4fa28-f262-44d7-9dd8-6a410dd7c51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:29.798041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4fa28-f262-44d7-9dd8-6a410dd7c51b service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:29.809709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1396/2777] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:30.378021 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:30.381243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:30.422816 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ef755-08b9-49e9-a197-9eafeacfd175 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:30.428845 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:30.429986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:30.436339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:30.436339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] PATH_INFO: `/v3/projects/624a87f068e74e94898c36b82fd19f6b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:30.436339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] RBAC: Authorizing `identity:delete_project(project_id=624a87f068e74e94898c36b82fd19f6b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:30.436339 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:30.622419 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624a87f068e74e94898c36b82fd19f6b', 'request_id': 'req-80948b2e-dba8-4266-aff9-48702d43b07b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:30.626459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:30.639559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624a87f068e74e94898c36b82fd19f6b', 'request_id': 'req-80948b2e-dba8-4266-aff9-48702d43b07b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:30.754983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624a87f068e74e94898c36b82fd19f6b', 'request_id': 'req-80948b2e-dba8-4266-aff9-48702d43b07b'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:30.776786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] The token cache is being invalidate because project 624a87f068e74e94898c36b82fd19f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:30.777398 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:30.778667 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-80948b2e-dba8-4266-aff9-48702d43b07b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:30.781531 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1382/2778] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:30 2022] DELETE /identity/v3/projects/624a87f068e74e94898c36b82fd19f6b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:30.789519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:30.790539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:30.790539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:30.824232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:30.826595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:30.869797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:30.957281 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1397/2779] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:30.965101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:30.965601 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:30.966056 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:30.997538 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:30.999060 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:31.019808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1383/2780] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:31.026781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:31.044764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db1f0f-33a8-4955-8b9c-9d731abcd044 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:31.054366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:31.055790 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.056333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.057131 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.060578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:35:31.061124 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:31.064311 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:31.089304 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1398/2781] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:35:31.095423 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:31.105988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adcd98-0bc5-4444-83f3-3862f55155ea None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:31.111428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:31.112146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.112461 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.112739 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.113476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:31.116121 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:31.258608 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1384/2782] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:35:31 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:35:31.265709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:31.292435 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ffd0c-fd3e-424d-93c8-53520fdf1e3f admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:31.397729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:31.398494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.406019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.406019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.406019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:31.406019 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:31.521367 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1399/2783] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:35:31 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:35:31.527314 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:31.545772 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5160e9-f2cc-4d31-b156-cafe5e631fba admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:31.553525 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:31.554657 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.555174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.555634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.556994 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:31.559392 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:31.575323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1385/2784] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:31.581983 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:31.605018 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687ac18-8742-4762-99bc-13d9a74da70b admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:31.611074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bc28b-9ca9-4858-8f47-46d11763ae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:31.612059 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bc28b-9ca9-4858-8f47-46d11763ae63 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.612531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bc28b-9ca9-4858-8f47-46d11763ae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.614293 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bc28b-9ca9-4858-8f47-46d11763ae63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.615252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bc28b-9ca9-4858-8f47-46d11763ae63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:31.617085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bc28b-9ca9-4858-8f47-46d11763ae63 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:31.635768 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1400/2785] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:35:31.642662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:31.655264 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb33e4-eb7a-4a34-bfb4-c55e4f56c3d1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:31.661396 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:31.662519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.663002 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.663428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] PATH_INFO: `/v3/projects/a3e40664a37c420c9fc2e05af20e182b/users/0d44567701db44e3802d73705c899dd4/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.684704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e40664a37c420c9fc2e05af20e182b, user_id=0d44567701db44e3802d73705c899dd4, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:31.687786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:31.847996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1386/2786] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:31 2022] PUT /identity/v3/projects/a3e40664a37c420c9fc2e05af20e182b/users/0d44567701db44e3802d73705c899dd4/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:31.858178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfadc4a-cda9-4918-bd85-2595654a8de5 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:31.859818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfadc4a-cda9-4918-bd85-2595654a8de5 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:31.860830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfadc4a-cda9-4918-bd85-2595654a8de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:31.902058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8cfadc4a-cda9-4918-bd85-2595654a8de5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:31.904186 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8cfadc4a-cda9-4918-bd85-2595654a8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:32.018354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1401/2787] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:32.047958 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:32.051964 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:32.209724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61c4b-ee6d-4036-9807-897430f87df1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:32.217558 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddba168-52a0-41d4-ad56-8a51ccac189b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:32.218691 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddba168-52a0-41d4-ad56-8a51ccac189b service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:32.219176 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddba168-52a0-41d4-ad56-8a51ccac189b service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:32.219736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddba168-52a0-41d4-ad56-8a51ccac189b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:32.233439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddba168-52a0-41d4-ad56-8a51ccac189b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:32.235549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddba168-52a0-41d4-ad56-8a51ccac189b service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:32.251343 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1387/2788] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:40.820698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:40.822191 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:40.822960 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:40.871813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:40.872646 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:40.872834 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:40.908710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:40.910576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:40.944742 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:40.959290 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] MFA Rules not processed for user `0d44567701db44e3802d73705c899dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:40.961462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:41.037898 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1402/2789] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:41.088736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1388/2790] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:41.094595 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:41.113244 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730041a-6711-415f-b342-c0fbdd8e947c None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:41.121545 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:41.122578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:41.123231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:41.123733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:41.125307 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:41.127945 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:41.254664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:41.257366 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1403/2791] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:35:41 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:35:41.281252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:41.283554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:41.285520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219a1f9-9851-42ca-ba6a-60bdd7304463 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:41.304854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:41.539561 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a23ed-5991-45df-ae65-09020f2c99cb admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:41.554422 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:41.556123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:41.556659 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:41.558709 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:41.568902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:41.570796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:41.574911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:41.575434 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] PATH_INFO: `/v3/users` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:41.576872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:41.580587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:41.698920 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1389/2792] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:35:41 2022] POST /identity/v3/users => generated 299 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:35:41.702668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:41.721395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c18fa5-6a15-4496-9c8e-5ab3c023d777 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:41.727719 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:41.729429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:41.731224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:41.731224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:41.731224 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:41.734386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:41.752334 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1390/2793] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 04 09:35:41.756450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:41.778644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8f85a-8e54-437c-bd0d-97a03a45d1dd admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:41.794190 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:41.794896 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:41.795119 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:41.795415 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:41.795701 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] PATH_INFO: `/v3/projects/a085348568874e83a4f22676311bae58/users/fd1f9cd8305448a7a0d80b6bc6657e6c/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:41.798518 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:41.820728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1404/2794] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:35:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:35:41.836650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a085348568874e83a4f22676311bae58, user_id=fd1f9cd8305448a7a0d80b6bc6657e6c, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:41.842316 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:41.996672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1391/2795] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:41 2022] PUT /identity/v3/projects/a085348568874e83a4f22676311bae58/users/fd1f9cd8305448a7a0d80b6bc6657e6c/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:42.001621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:42.028269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9aa29-d506-4d81-9626-bcd497565ead service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:42.155823 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:42.182778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:42.182778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:42.182778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:42.182778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:42.183123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:42.185145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:42.208411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1405/2796] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:35:42.212171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:42.230763 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f0c59-9bff-4bf0-a355-823b555f147d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:42.239127 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934df05-d744-4182-a880-60df4fee79ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:42.240145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934df05-d744-4182-a880-60df4fee79ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:42.240609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934df05-d744-4182-a880-60df4fee79ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:42.240732 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934df05-d744-4182-a880-60df4fee79ac admin admin] PATH_INFO: `/v3/projects/a085348568874e83a4f22676311bae58/users/fd1f9cd8305448a7a0d80b6bc6657e6c/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:42.243501 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934df05-d744-4182-a880-60df4fee79ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a085348568874e83a4f22676311bae58, user_id=fd1f9cd8305448a7a0d80b6bc6657e6c, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:42.246814 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934df05-d744-4182-a880-60df4fee79ac admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:42.263329 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1392/2797] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:35:42 2022] PUT /identity/v3/projects/a085348568874e83a4f22676311bae58/users/fd1f9cd8305448a7a0d80b6bc6657e6c/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:43.477825 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:43.649853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7079e-f5bf-4a97-82d7-df4b9b623e1d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:43.758129 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:43.761605 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:43.761924 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:43.762192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:43.773369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:43.928999 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:43.930808 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:43.945726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1406/2798] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:45.767446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bae8c-6055-4698-b333-24bf3a942c1b None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:45.768585 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bae8c-6055-4698-b333-24bf3a942c1b None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:45.769294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bae8c-6055-4698-b333-24bf3a942c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:45.810996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-f00bae8c-6055-4698-b333-24bf3a942c1b None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:45.813437 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-f00bae8c-6055-4698-b333-24bf3a942c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:45.966988 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1393/2799] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:35:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:46.015040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:46.388566 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:46.422875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d9c95-dd0a-4d5a-92e4-7f722bb541d6 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:46.441950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:46.443036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:46.444819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:46.447004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:46.487927 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:46.492251 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:46.526934 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1407/2800] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:35:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:35:48.939628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59e114-49be-4811-ac35-7e160add9c95 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:48.940474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59e114-49be-4811-ac35-7e160add9c95 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:48.940474 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59e114-49be-4811-ac35-7e160add9c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:48.985096 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-3a59e114-49be-4811-ac35-7e160add9c95 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:48.986418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-3a59e114-49be-4811-ac35-7e160add9c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:49.009729 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1394/2801] 10.222.0.46 () {58 vars in 967 bytes} [Sat Jun 4 09:35:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:49.041427 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:49.087940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044ad38-f5ec-4a66-b3b8-d21989329f1e service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:49.100411 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:49.102112 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:49.103267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:49.103925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:49.138362 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:49.142657 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:49.200231 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1408/2802] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:35:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:35:50.653904 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e059d-23ee-4d15-b5ca-40bc15a268dd None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:50.654807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e059d-23ee-4d15-b5ca-40bc15a268dd None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:50.654807 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e059d-23ee-4d15-b5ca-40bc15a268dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:50.725126 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b01e059d-23ee-4d15-b5ca-40bc15a268dd None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:50.726639 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b01e059d-23ee-4d15-b5ca-40bc15a268dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:50.748239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1395/2803] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:35:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:50.769429 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:50.795150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409c57-2ab8-423f-b2d1-80a09c659d34 service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:50.818564 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:50.820369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:50.821150 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:50.821639 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:50.849166 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:50.857731 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:50.891243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1409/2804] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:35:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:35:52.531323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad8de-3c11-49c5-b97a-bd17b6b1e1ec None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:52.532130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad8de-3c11-49c5-b97a-bd17b6b1e1ec None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:52.532610 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad8de-3c11-49c5-b97a-bd17b6b1e1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:52.640694 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-467ad8de-3c11-49c5-b97a-bd17b6b1e1ec None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:52.642859 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-467ad8de-3c11-49c5-b97a-bd17b6b1e1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:52.701099 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1396/2805] 10.222.0.46 () {58 vars in 967 bytes} [Sat Jun 4 09:35:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:52.722537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:52.767092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b1be-7cf5-4e35-91c9-b4bd7c0aebf9 service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:52.788541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:52.790115 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:52.791211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:52.791948 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:52.833358 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:52.838117 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:52.880816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1410/2806] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:35:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:35:57.400549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:57.401318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:57.401797 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:57.447749 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] MFA Rules not processed for user `fd1f9cd8305448a7a0d80b6bc6657e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:57.461715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:57.535571 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:57.574842 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:57.576275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:57.659547 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1397/2807] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:57.675296 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:57.679294 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:57.766952 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af52b-2e1d-4a0b-aad1-521350467bd7 service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:57.788062 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:57.789598 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:57.790105 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:57.790616 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] PATH_INFO: `/v3/users/8f357411d6a547468e94697978f9ca6f` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:57.798443 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f357411d6a547468e94697978f9ca6f)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:57.802249 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:57.882085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea9e1-fbfc-4470-98c3-b94f400b45f0 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:57.895887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a75be-298d-4a74-b844-d301cb64f6a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3a9e1e2e3d438f92286a4158c8d4cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:57.897656 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a75be-298d-4a74-b844-d301cb64f6a8 service glance] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:57.904137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a75be-298d-4a74-b844-d301cb64f6a8 service glance] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:57.905587 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a75be-298d-4a74-b844-d301cb64f6a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:58.059548 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a75be-298d-4a74-b844-d301cb64f6a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:58.061707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a75be-298d-4a74-b844-d301cb64f6a8 service glance] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:58.084530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1398/2808] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:35:58.108519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f357411d6a547468e94697978f9ca6f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:58.128144 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f357411d6a547468e94697978f9ca6f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:58.130309 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f357411d6a547468e94697978f9ca6f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:58.161749 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f357411d6a547468e94697978f9ca6f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:58.164257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f357411d6a547468e94697978f9ca6f'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:58.326867 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1411/2809] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:35:57 2022] DELETE /identity/v3/users/8f357411d6a547468e94697978f9ca6f => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:35:58.333949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa172-8f18-45ad-9b2e-b6a79ba7dec6 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:58.334612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa172-8f18-45ad-9b2e-b6a79ba7dec6 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:58.335120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa172-8f18-45ad-9b2e-b6a79ba7dec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:58.383476 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b1ffa172-8f18-45ad-9b2e-b6a79ba7dec6 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:58.386742 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b1ffa172-8f18-45ad-9b2e-b6a79ba7dec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:58.505676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1399/2810] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:35:58.521176 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:58.523159 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:58.591613 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:58.592736 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:58.594037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:58.594037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:58.627748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:58.630664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:58.646545 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:35:58.650500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce8c3-f983-4caa-b49e-bcdcf99f3b93 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:58.655516 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:58.656546 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:58.656925 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:58.657302 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:58.685143 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:58.689724 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:58.707920 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1412/2811] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:35:58.731428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1400/2812] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:35:58.771977 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:58.774799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:58.955283 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9a976-a4b1-4940-8856-f0d9bc7519de service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:58.964326 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95d750-2eb8-45d3-9bfb-0f772e88560b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:58.965804 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95d750-2eb8-45d3-9bfb-0f772e88560b service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:58.966660 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95d750-2eb8-45d3-9bfb-0f772e88560b service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:58.967252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95d750-2eb8-45d3-9bfb-0f772e88560b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:58.995587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95d750-2eb8-45d3-9bfb-0f772e88560b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:58.998988 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95d750-2eb8-45d3-9bfb-0f772e88560b service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:59.026325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1413/2813] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:35:59.224812 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:35:59.227533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:35:59.279760 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3717b-6741-49f8-8c81-85a0304a5938 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:35:59.291338 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:35:59.292848 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:59.293369 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:59.293953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] PATH_INFO: `/v3/projects/ebe772f995f743118963aa8c85ab5c2b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:59.296973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe772f995f743118963aa8c85ab5c2b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:35:59.327187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:35:59.592718 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe772f995f743118963aa8c85ab5c2b', 'request_id': 'req-8dfe6028-7cd8-48bd-967c-942dc4b2387f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:59.648253 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe772f995f743118963aa8c85ab5c2b', 'request_id': 'req-8dfe6028-7cd8-48bd-967c-942dc4b2387f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:59.748228 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe772f995f743118963aa8c85ab5c2b', 'request_id': 'req-8dfe6028-7cd8-48bd-967c-942dc4b2387f'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:59.784636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] The token cache is being invalidate because project ebe772f995f743118963aa8c85ab5c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:35:59.785881 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:59.787285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-8dfe6028-7cd8-48bd-967c-942dc4b2387f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:35:59.794853 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1401/2814] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:35:59 2022] DELETE /identity/v3/projects/ebe772f995f743118963aa8c85ab5c2b => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:35:59.800142 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:35:59.801739 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:35:59.802272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:35:59.888446 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:35:59.890549 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:00.052352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1414/2815] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:00.059055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.059055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.059438 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:00.087118 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:00.088351 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:00.102355 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1402/2816] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:36:00.103340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:00.115178 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5527-f39c-42dc-b6a9-054de8ef611e None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:00.121046 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:00.122630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.122942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.123230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:00.124023 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:36:00.124369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:00.126781 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:00.142837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1415/2817] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 04 09:36:00.147702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:00.166727 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a02b-9e7c-4f38-ad7c-9a7bb6d14c56 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:00.174748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:00.175549 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.175837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.176115 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:00.176901 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:00.179246 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:00.459137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1403/2818] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:36:00 2022] POST /identity/v3/projects => generated 383 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:36:00.466745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:00.504377 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d29d6-2103-4300-b76e-8b10d07ae2e3 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:00.570408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=100 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:00.643764 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:00.647906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.647906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.654498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:00.655676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:00.671989 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:00.809937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:00.812636 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1416/2819] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:36:00 2022] POST /identity/v3/users => generated 292 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:36:00.827459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100385-1dd2-4ead-8566-86eda1c4604a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:00.836403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:00.837430 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.837867 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.838292 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:00.839243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:00.842407 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:00.865368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1404/2820] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:36:00.867208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:00.889646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fef8d-44cd-46d9-b9a3-df2c12b85b1a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:00.903526 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593fbeb-6056-491e-acb7-bf6f81226d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:00.904665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593fbeb-6056-491e-acb7-bf6f81226d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.908968 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593fbeb-6056-491e-acb7-bf6f81226d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.909685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593fbeb-6056-491e-acb7-bf6f81226d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:00.911793 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593fbeb-6056-491e-acb7-bf6f81226d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:00.921395 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593fbeb-6056-491e-acb7-bf6f81226d36 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:00.957852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1417/2821] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:36:00.960045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:00.980540 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821f15-919c-483f-84fc-beca7568ec3c admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:00.993378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:00.995921 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:00.996666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:00.997311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] PATH_INFO: `/v3/projects/77242525fe2f49c6b38755c270885b79/users/611fcce3edba4ce68c50d96736c7b853/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:01.044481 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=77242525fe2f49c6b38755c270885b79, user_id=611fcce3edba4ce68c50d96736c7b853, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:01.070888 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:01.179715 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1405/2822] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:36:00 2022] PUT /identity/v3/projects/77242525fe2f49c6b38755c270885b79/users/611fcce3edba4ce68c50d96736c7b853/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:36:01.182241 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc76a2c-cc7c-40ef-8c98-37a20b56cdc2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:01.182819 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc76a2c-cc7c-40ef-8c98-37a20b56cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:01.183397 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc76a2c-cc7c-40ef-8c98-37a20b56cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:01.242080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-8cc76a2c-cc7c-40ef-8c98-37a20b56cdc2 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:01.250879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-8cc76a2c-cc7c-40ef-8c98-37a20b56cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:01.571710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1418/2823] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:01.590782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:01.592936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:01.740422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:01.778544 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dad425-39d5-4264-977e-d260346b8b7d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:01.797666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:01.798359 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:01.798644 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:01.798922 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:01.838089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:01.839622 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:01.889591 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1406/2824] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:36:05.377287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fa562-31ba-4cc9-9891-cef2030df93d None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:05.377287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fa562-31ba-4cc9-9891-cef2030df93d None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:05.377287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fa562-31ba-4cc9-9891-cef2030df93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:05.421837 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-664fa562-31ba-4cc9-9891-cef2030df93d None None] MFA Rules not processed for user `611fcce3edba4ce68c50d96736c7b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:05.424687 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-664fa562-31ba-4cc9-9891-cef2030df93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:05.667346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1419/2825] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:36:05.686146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:05.688641 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:05.913825 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc5cd4-4a24-4d6e-9639-243dfd66d052 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:05.926768 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407c86-f20c-4bd5-98e0-2a8251105601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:05.928186 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407c86-f20c-4bd5-98e0-2a8251105601 service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:05.928972 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407c86-f20c-4bd5-98e0-2a8251105601 service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:05.933275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407c86-f20c-4bd5-98e0-2a8251105601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:05.951917 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407c86-f20c-4bd5-98e0-2a8251105601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:05.954983 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407c86-f20c-4bd5-98e0-2a8251105601 service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:05.985634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1407/2826] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:36:08.562593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:08.563778 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:08.564259 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:08.598076 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] MFA Rules not processed for user `611fcce3edba4ce68c50d96736c7b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:08.599853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:08.639152 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1420/2827] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:13.671953 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:13.672943 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:13.673111 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:13.698728 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:13.699897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:13.784033 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1408/2828] 10.222.0.46 () {58 vars in 967 bytes} [Sat Jun 4 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:13.806189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:13.807879 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:13.875347 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:13.905908 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ce17c-34f3-4044-878a-e88c00ec7ead None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:13.911472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:13.912214 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:13.912573 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:13.912895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:13.922511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:13.924581 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:13.935538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1421/2829] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:36:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 04 09:36:14.678275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:14.679803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:14.694662 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6060c-8a91-4014-a337-5d703471920e None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:14.701164 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:14.701976 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:14.702368 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:14.703249 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:14.704883 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:14.840462 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:14.845511 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c0720-0fcf-4f84-9686-02d2a7783d88 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:14.864055 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1409/2830] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:36:16.006758 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:16.017733 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd08ac-ce76-4e52-b837-9aec7b5e6e80 service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:16.022095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:16.022812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:16.023232 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:16.023621 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:16.025871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:16.043766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:16.045288 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:16.059456 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1422/2831] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:36:16.499453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:16.500137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:16.500137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:16.520405 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] MFA Rules not processed for user `0d44567701db44e3802d73705c899dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:16.521902 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:16.532482 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=100 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:16.598802 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1410/2832] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:36:19.917662 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:19.919571 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:19.936798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0b0a6-f6ae-4303-a316-800638b2c804 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:19.943172 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:19.946353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:19.946353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:19.946353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] PATH_INFO: `/v3/users/078a0f5285624b5f82b80e53a3a68ef9` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:19.946353 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] RBAC: Authorizing `identity:delete_user(user_id=078a0f5285624b5f82b80e53a3a68ef9)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:19.948407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:20.286590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078a0f5285624b5f82b80e53a3a68ef9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:20.304708 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078a0f5285624b5f82b80e53a3a68ef9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:20.305883 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078a0f5285624b5f82b80e53a3a68ef9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:20.329271 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078a0f5285624b5f82b80e53a3a68ef9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:20.331252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-2b8957f9-d112-42f6-ad23-ee92909d001c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078a0f5285624b5f82b80e53a3a68ef9'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:20.491414 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1423/2833] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:36:19 2022] DELETE /identity/v3/users/078a0f5285624b5f82b80e53a3a68ef9 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:36:20.811260 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:20.838120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49ee79-9214-41b3-9e26-8720f1fb9cfa None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:20.974705 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:20.979038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:20.979038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:20.979038 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] PATH_INFO: `/v3/projects/fff077a9afa543b2b239071f9c79ac38` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:20.980318 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff077a9afa543b2b239071f9c79ac38)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:20.985230 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:21.175113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff077a9afa543b2b239071f9c79ac38', 'request_id': 'req-548cbe95-9232-4c3d-a951-77e0cefbc655'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:21.184803 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff077a9afa543b2b239071f9c79ac38', 'request_id': 'req-548cbe95-9232-4c3d-a951-77e0cefbc655'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:21.290582 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff077a9afa543b2b239071f9c79ac38', 'request_id': 'req-548cbe95-9232-4c3d-a951-77e0cefbc655'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:21.309781 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] The token cache is being invalidate because project fff077a9afa543b2b239071f9c79ac38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:36:21.310269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:21.311454 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-548cbe95-9232-4c3d-a951-77e0cefbc655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:21.316020 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1411/2834] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:36:20 2022] DELETE /identity/v3/projects/fff077a9afa543b2b239071f9c79ac38 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:36:21.320894 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71b36c-cf66-4e2d-a3b7-232788446109 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.321539 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71b36c-cf66-4e2d-a3b7-232788446109 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.321792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71b36c-cf66-4e2d-a3b7-232788446109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.360572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-9b71b36c-cf66-4e2d-a3b7-232788446109 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:21.362715 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9b71b36c-cf66-4e2d-a3b7-232788446109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:21.418066 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.418554 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.419046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.450567 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:21.456580 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:21.507247 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1424/2835] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:36:21.520235 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.520722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.521101 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.550605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:21.557966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:21.560324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:21.572823 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1412/2836] 10.222.0.46 () {58 vars in 1009 bytes} [Sat Jun 4 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:36:21.581303 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1425/2837] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:21.583724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:21.595068 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:21.598096 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:21.601165 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4071cd-6906-4131-b166-f27591879c50 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:21.607493 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:21.608418 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.608813 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.610163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.610633 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:36:21.611052 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:21.615682 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:21.640244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1413/2838] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:36:21.642089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:21.656109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4636-4717-47ea-905d-db7c1a541f4a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:21.665680 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:21.668004 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.668439 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.668906 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.670145 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:21.673086 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:21.683537 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:21.745394 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112f8b8-ed3f-4e9a-b4c8-b9723721ba10 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:21.754179 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:21.755205 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.755630 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.755958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.773445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:21.773445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:21.784950 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1426/2839] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:36:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:36:21.852794 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1414/2840] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:36:21 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:36:21.859585 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:21.878913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5353a-cd45-4817-9960-18ad385510e6 service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:21.978533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:21.979351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:21.979813 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:21.980331 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:21.981276 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:21.985285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:22.115835 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1427/2841] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:36:21 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:36:22.123130 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:22.166006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba71fb8-2b08-4712-a3d2-4c7af11fa98d admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:22.187879 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:22.189978 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:22.191871 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:22.193081 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:22.198533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:22.203076 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:22.244492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1415/2842] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:36:22.251745 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:22.287499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15603-a0f7-44f3-b2c7-ae73619d141d admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:22.309743 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e58ad-94f7-476f-9c5a-cf419b763451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:22.319412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e58ad-94f7-476f-9c5a-cf419b763451 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:22.319412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e58ad-94f7-476f-9c5a-cf419b763451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:22.319412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e58ad-94f7-476f-9c5a-cf419b763451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:22.319412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e58ad-94f7-476f-9c5a-cf419b763451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:22.319412 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e58ad-94f7-476f-9c5a-cf419b763451 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:22.354475 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1428/2843] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:36:22.360327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:22.380743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ce2e-4334-42f9-b081-84d5f7918024 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:22.390244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:22.393741 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:22.394671 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:22.395458 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] PATH_INFO: `/v3/projects/a7f190f929314196b87d05db15266e6c/users/eaa1ab2e75634a6f90e4c8f3c7be51c2/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:22.435131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f190f929314196b87d05db15266e6c, user_id=eaa1ab2e75634a6f90e4c8f3c7be51c2, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:22.442155 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:22.567429 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1416/2844] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:36:22 2022] PUT /identity/v3/projects/a7f190f929314196b87d05db15266e6c/users/eaa1ab2e75634a6f90e4c8f3c7be51c2/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:36:22.582279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:22.582965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:22.583729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:22.639853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] MFA Rules not processed for user `eaa1ab2e75634a6f90e4c8f3c7be51c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:22.642519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:22.817987 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1429/2845] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:22.824101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:22.826324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:23.091431 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ccd9-e331-4051-ba6a-b2bf4fdf7dc1 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:23.109466 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a29d1-7388-4f68-b086-9dbaf053e61d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e53e659f7faf403b9f33f393d166dda3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:23.109937 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a29d1-7388-4f68-b086-9dbaf053e61d service cinder] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:23.110226 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a29d1-7388-4f68-b086-9dbaf053e61d service cinder] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:23.110506 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a29d1-7388-4f68-b086-9dbaf053e61d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:23.126762 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a29d1-7388-4f68-b086-9dbaf053e61d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:23.128939 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a29d1-7388-4f68-b086-9dbaf053e61d service cinder] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:23.161740 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1417/2846] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:36:32.803576 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:33.143666 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:38.789832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:38.871061 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2539352-d99d-400c-995d-2fbc7e92f7b7 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:38.876181 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:38.877200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:38.877486 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:38.877766 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] PATH_INFO: `/v3/users/eaa1ab2e75634a6f90e4c8f3c7be51c2` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:38.879222 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa1ab2e75634a6f90e4c8f3c7be51c2)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:38.881243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:39.137881 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa1ab2e75634a6f90e4c8f3c7be51c2'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:39.146902 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa1ab2e75634a6f90e4c8f3c7be51c2'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:39.147676 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa1ab2e75634a6f90e4c8f3c7be51c2'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:39.163492 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa1ab2e75634a6f90e4c8f3c7be51c2'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:39.164631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa1ab2e75634a6f90e4c8f3c7be51c2'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:39.283407 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1430/2847] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:36:38 2022] DELETE /identity/v3/users/eaa1ab2e75634a6f90e4c8f3c7be51c2 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:36:39.289860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:39.290832 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:39.291035 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:39.313268 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:36:39.314643 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:39.411285 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1418/2848] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:36:39.421935 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:39.424811 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:39.510821 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4003e-f59d-4ed5-b8e0-e27e443f64b5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:39.515092 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:39.515818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:39.516118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:39.516481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:39.528973 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:39.530801 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:39.543984 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1431/2849] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:36:40.179053 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:40.181380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:40.228672 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e363e5-a8bc-41d3-84dc-1211ebfe59c5 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:40.232205 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:40.234108 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:40.234594 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:40.235101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] PATH_INFO: `/v3/projects/a7f190f929314196b87d05db15266e6c` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:40.238134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f190f929314196b87d05db15266e6c)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:40.244233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:40.442298 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f190f929314196b87d05db15266e6c', 'request_id': 'req-02a30329-56a2-4b3a-b1c1-7ac30b120037'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:40.458669 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f190f929314196b87d05db15266e6c', 'request_id': 'req-02a30329-56a2-4b3a-b1c1-7ac30b120037'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:40.580675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f190f929314196b87d05db15266e6c', 'request_id': 'req-02a30329-56a2-4b3a-b1c1-7ac30b120037'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:40.601031 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] The token cache is being invalidate because project a7f190f929314196b87d05db15266e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:36:40.601631 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:40.602800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:36:40.607912 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1419/2850] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:36:40 2022] DELETE /identity/v3/projects/a7f190f929314196b87d05db15266e6c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:36:49.537890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:50.593471 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:36:53.040499 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:36:53.053257 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4cd73-bc82-4705-abd0-f23bd1d51f03 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:36:53.130137 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:36:53.130818 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:36:53.131103 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:36:53.131379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:36:53.133533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:36:53.209380 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:36:53.210947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:36:53.223685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1432/2851] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:36:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:37:01.047675 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:37:01.050530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:01.157898 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30329-56a2-4b3a-b1c1-7ac30b120037 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:37:01.164820 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:37:01.165503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:37:01.165789 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:37:01.166069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:37:01.169725 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:01.189611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:37:01.195484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304156c-b819-4c10-9ee4-b807c0f51b96 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:37:01.234317 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1420/2852] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:37:03.217406 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=50 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:37:11.211277 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=34 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:37:19.329295 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:37:19.331485 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:19.343798 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f91c69-38c3-4db9-9960-342f9087c999 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:37:19.348531 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:37:19.349324 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:37:19.349655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:37:19.350074 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:37:19.351997 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:19.364364 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:37:19.365895 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:37:19.375524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1433/2853] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:37:29.371316 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:37:36.322822 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:37:36.323492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:37:36.323492 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:37:36.355940 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] MFA Rules not processed for user `611fcce3edba4ce68c50d96736c7b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:37:36.357380 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:36.451586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1421/2854] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:37:36.731603 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:37:36.757342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c588f5-8ef3-49b6-a2ef-f2a04c4d8f35 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:37:36.761871 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:37:36.762596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:37:36.762929 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:37:36.763274 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:37:36.765436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:36.791519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:37:36.793027 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:37:36.803614 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1434/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:37:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:37:37.443860 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:37:37.445617 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:37.567865 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678adc7b-c0a8-4a2a-b46b-feaedcb85acf None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:37:37.576273 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:37:37.577393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:37:37.577866 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:37:37.578363 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:37:37.581959 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:37.603262 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:37:37.605586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:37:37.619435 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1422/2856] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:37:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:37:46.798509 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=15 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:37:47.613331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=72 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:37:48.775535 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:37:48.796645 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb9db-af0d-4318-ba57-d679697e0d2e service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:37:48.803014 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:37:48.803327 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:37:48.803726 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:37:48.804089 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:37:48.806269 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:37:48.820812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:37:48.822417 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:37:48.835697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1435/2857] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:37:58.830413 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:00.647112 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:00.678071 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8f4a2-a618-4654-aee4-3466933038c2 service placement] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:00.682660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:00.683382 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:00.683674 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:00.684023 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:00.686559 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:00.732539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:00.735779 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb01093-1097-479c-8ba8-66a2ee846890 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:00.752778 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1423/2858] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:38:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:38:00.835918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:00.847383 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205bb0f-5775-402b-8505-44471f315fec service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:00.853083 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:00.854215 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:00.854719 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:00.855246 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:00.858792 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:00.887008 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:00.889928 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:00.914632 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1436/2859] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:38:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:38:10.744844 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=15 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:10.903500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:32.512275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:32.518986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:32.518986 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:32.533611 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] MFA Rules not processed for user `d1bb3c381d7c47aab9d79db14debc319`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:38:32.534872 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:32.602792 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1424/2860] 10.222.0.46 () {58 vars in 967 bytes} [Sat Jun 4 09:38:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:38:32.661344 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:32.662942 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:32.747000 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781ed69-496c-45b7-a29e-97f79a5ff620 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:32.752590 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a16b32249c48eb89cf0d7cd55e5de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:32.753258 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:32.753644 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:32.753930 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:32.763360 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:32.765084 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:32.774966 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1437/2861] 10.222.0.46 () {60 vars in 1384 bytes} [Sat Jun 4 09:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 04 09:38:38.502397 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:38.515137 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9fb5-695f-4fd9-895a-a4b8e647a11a None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:38.520858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:38.521530 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:38.521819 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:38.522104 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:38.527185 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:38.546427 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:38.550015 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:38.568170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1425/2862] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:38:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:38:42.771524 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=32 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:43.435584 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:43.439520 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:43.572504 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759d16-a48c-41e8-9e70-0a8713d7839d service swift] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:43.579230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:43.580699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:43.581130 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:43.581707 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] PATH_INFO: `/v3/users/0d44567701db44e3802d73705c899dd4` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:43.583922 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d44567701db44e3802d73705c899dd4)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:43.588048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:43.856469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d44567701db44e3802d73705c899dd4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:43.868553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d44567701db44e3802d73705c899dd4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:43.868553 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d44567701db44e3802d73705c899dd4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:43.882134 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d44567701db44e3802d73705c899dd4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:43.882954 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d44567701db44e3802d73705c899dd4'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:43.955890 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1438/2863] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:38:43 2022] DELETE /identity/v3/users/0d44567701db44e3802d73705c899dd4 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:38:44.243503 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:44.262109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d44bb-3ee4-4033-80a1-1635b2b1450c service placement] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:44.342874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:44.343847 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:44.344215 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:44.344598 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] PATH_INFO: `/v3/projects/a3e40664a37c420c9fc2e05af20e182b` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:44.346040 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e40664a37c420c9fc2e05af20e182b)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:44.348122 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:44.552512 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e40664a37c420c9fc2e05af20e182b', 'request_id': 'req-de72b3d0-51d3-4220-8cb3-7e70bffc3979'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:44.559936 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e40664a37c420c9fc2e05af20e182b', 'request_id': 'req-de72b3d0-51d3-4220-8cb3-7e70bffc3979'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:44.663251 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e40664a37c420c9fc2e05af20e182b', 'request_id': 'req-de72b3d0-51d3-4220-8cb3-7e70bffc3979'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:44.678034 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] The token cache is being invalidate because project a3e40664a37c420c9fc2e05af20e182b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:38:44.678393 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:44.679078 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:44.680769 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1426/2864] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:38:44 2022] DELETE /identity/v3/projects/a3e40664a37c420c9fc2e05af20e182b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:38:44.685102 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:44.686812 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:44.704901 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf9fec-f464-47a9-b5cf-9719bbd83d67 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:44.772465 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:44.784860 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:44.785757 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:44.786044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:44.786346 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] PATH_INFO: `/v3/users/fd1f9cd8305448a7a0d80b6bc6657e6c` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:44.787655 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1f9cd8305448a7a0d80b6bc6657e6c)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:44.789728 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:45.093058 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1f9cd8305448a7a0d80b6bc6657e6c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.101853 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1f9cd8305448a7a0d80b6bc6657e6c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.102684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1f9cd8305448a7a0d80b6bc6657e6c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.122391 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1f9cd8305448a7a0d80b6bc6657e6c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.125583 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-a10874d5-e817-4124-882d-33922f313e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1f9cd8305448a7a0d80b6bc6657e6c'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.219759 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1439/2865] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:38:44 2022] DELETE /identity/v3/users/fd1f9cd8305448a7a0d80b6bc6657e6c => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:38:45.562647 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:45.564616 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:45.580543 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b3d0-51d3-4220-8cb3-7e70bffc3979 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:45.610609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:45.678446 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:45.679516 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:45.679981 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:45.680352 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] PATH_INFO: `/v3/projects/a085348568874e83a4f22676311bae58` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:45.682131 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a085348568874e83a4f22676311bae58)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:45.684335 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:45.849272 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a085348568874e83a4f22676311bae58', 'request_id': 'req-cc3ed3bf-8abb-4371-94df-51afff76cefc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.856171 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a085348568874e83a4f22676311bae58', 'request_id': 'req-cc3ed3bf-8abb-4371-94df-51afff76cefc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.931748 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a085348568874e83a4f22676311bae58', 'request_id': 'req-cc3ed3bf-8abb-4371-94df-51afff76cefc'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.945366 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] The token cache is being invalidate because project a085348568874e83a4f22676311bae58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:38:45.945664 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.946311 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cc3ed3bf-8abb-4371-94df-51afff76cefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:45.947911 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1427/2866] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:38:45 2022] DELETE /identity/v3/projects/a085348568874e83a4f22676311bae58 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:38:45.954141 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:45.954436 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:45.954696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.026196 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:38:46.028354 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:46.106208 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1440/2867] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:38:46.114973 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.115371 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.115724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.139552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:38:46.141505 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:46.150751 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1428/2868] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:38:46.157476 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.172188 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec5515-327a-4bcc-9c55-54315320534f None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:46.180761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:46.181599 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.182053 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.182511 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] PATH_INFO: `/v3/domains` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.183500 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 04 09:38:46.183875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:46.185859 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:46.200125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1441/2869] 10.222.0.46 () {58 vars in 1152 bytes} [Sat Jun 4 09:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 04 09:38:46.204653 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.215042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67fa8e-705a-4e4f-b613-6781295b8769 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:46.220362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:46.221054 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.221336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.221620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.222327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:46.224402 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:46.350642 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1429/2870] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:38:46 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:38:46.355753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.368055 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3ca0-c790-4f85-811b-6f6c120dd18a admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:46.396720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:46.446939 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:46.450772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.450772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.450772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.450772 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:46.451416 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:46.573799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1442/2871] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:38:46 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:38:46.579300 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.599377 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce83fb8-c108-41e2-bc7f-8586a7524867 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:46.607786 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:46.609632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.610619 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.610891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.612374 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:46.615552 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:46.645092 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1430/2872] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:38:46.645351 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.662481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462d6d-3e5e-4dbf-8791-558befb15a19 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:46.671095 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d3120-5f24-47cd-98b4-8a4cd5c9d152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:46.672875 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d3120-5f24-47cd-98b4-8a4cd5c9d152 admin admin] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.673363 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d3120-5f24-47cd-98b4-8a4cd5c9d152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.674192 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d3120-5f24-47cd-98b4-8a4cd5c9d152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.675469 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d3120-5f24-47cd-98b4-8a4cd5c9d152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:46.677507 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d3120-5f24-47cd-98b4-8a4cd5c9d152 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:46.691040 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1443/2873] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:38:46.696005 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.711485 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126bafd-de4e-4234-80f6-1109f5025c57 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:46.717717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:46.719697 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.720142 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.720614 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] PATH_INFO: `/v3/projects/3c5c8695d627407f91f991dc91808a4b/users/c59c71bc492841b3b22c8e6d66599768/roles/60342ac4b2b144e98d66f95f23780594` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.739327 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5c8695d627407f91f991dc91808a4b, user_id=c59c71bc492841b3b22c8e6d66599768, role_id=60342ac4b2b144e98d66f95f23780594)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:46.742270 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:46.825045 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1431/2874] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:38:46 2022] PUT /identity/v3/projects/3c5c8695d627407f91f991dc91808a4b/users/c59c71bc492841b3b22c8e6d66599768/roles/60342ac4b2b144e98d66f95f23780594 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:38:46.831720 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:46.832252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:46.832975 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:46.865228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] MFA Rules not processed for user `8d4e8588f0b448199c8c5e35afe88ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:38:46.867586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:46.976372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1444/2875] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:38:46.987065 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:46.989026 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:47.050907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:47.055072 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7c01-4a2e-4bef-b00d-72c24f2d5076 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:47.059236 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:47.059899 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:47.060188 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:47.064386 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:47.071109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:47.072864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:47.084897 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1432/2876] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:38:52.736889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:52.739233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:52.754029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f676fc-52b4-434e-a6c4-30f72ab38711 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:52.762081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:52.764533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:52.764533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:52.764533 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] PATH_INFO: `/v3/users/611fcce3edba4ce68c50d96736c7b853` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:52.767378 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=611fcce3edba4ce68c50d96736c7b853)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:52.769477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:53.040320 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '611fcce3edba4ce68c50d96736c7b853'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:53.055004 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '611fcce3edba4ce68c50d96736c7b853'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:53.057182 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '611fcce3edba4ce68c50d96736c7b853'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:53.080906 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '611fcce3edba4ce68c50d96736c7b853'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:53.081684 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-13344e2d-d134-4ba2-8d43-13126cbac085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '611fcce3edba4ce68c50d96736c7b853'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:53.193799 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1445/2877] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:38:52 2022] DELETE /identity/v3/users/611fcce3edba4ce68c50d96736c7b853 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:38:53.565447 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:53.590080 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89513ed8-3d48-437c-a10d-b391edaa71c5 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:53.707743 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:53.708464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:53.708944 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:53.709612 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] PATH_INFO: `/v3/projects/77242525fe2f49c6b38755c270885b79` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:53.712050 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77242525fe2f49c6b38755c270885b79)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:53.716197 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:53.925874 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77242525fe2f49c6b38755c270885b79', 'request_id': 'req-a23f9114-89c2-452e-8aa5-a9209f68c731'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:53.935478 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77242525fe2f49c6b38755c270885b79', 'request_id': 'req-a23f9114-89c2-452e-8aa5-a9209f68c731'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:54.092027 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77242525fe2f49c6b38755c270885b79', 'request_id': 'req-a23f9114-89c2-452e-8aa5-a9209f68c731'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:54.111919 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] The token cache is being invalidate because project 77242525fe2f49c6b38755c270885b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:38:54.112457 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:54.113668 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:38:54.116378 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1433/2878] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:38:53 2022] DELETE /identity/v3/projects/77242525fe2f49c6b38755c270885b79 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:38:54.525325 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:54.526216 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:54.526463 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:54.567454 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] MFA Rules not processed for user `c59c71bc492841b3b22c8e6d66599768`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:38:54.568882 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:54.574816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:54.661554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1446/2879] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:38:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:38:54.668450 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:54.670011 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:38:54.757089 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f9114-89c2-452e-8aa5-a9209f68c731 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:54.762214 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:54.762905 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:54.763182 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:54.763615 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:54.764390 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:54.766464 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:54.906519 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1434/2880] 10.222.0.46 () {60 vars in 1152 bytes} [Sat Jun 4 09:38:54 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 04 09:38:54.910782 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:54.922275 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1027ab-6f64-4992-abc0-331d50e70341 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:54.986933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:54.987633 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:54.987921 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:54.988200 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] PATH_INFO: `/v3/users` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:54.988974 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:54.990965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:55.084116 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1447/2881] 10.222.0.46 () {60 vars in 1143 bytes} [Sat Jun 4 09:38:54 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 04 09:38:55.085891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:55.101605 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b374b-25e5-4917-acf3-71ae124d73ed admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:55.106777 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:55.107796 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:55.108085 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:55.108403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:55.109174 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:55.110869 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:55.113345 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:38:55.127101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1435/2882] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:38:55.132267 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:55.153243 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e109-ab42-4ad8-aab5-cf62737a8d77 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:55.161449 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:55.163087 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:55.163578 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:55.164154 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] PATH_INFO: `/v3/projects/5e85e2e369fa4f9c921eeb851b45e21e/users/0d1a55feebef4daa94a44673ecec33a7/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:55.198854 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e85e2e369fa4f9c921eeb851b45e21e, user_id=0d1a55feebef4daa94a44673ecec33a7, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:55.203184 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:55.287202 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1448/2883] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:38:55 2022] PUT /identity/v3/projects/5e85e2e369fa4f9c921eeb851b45e21e/users/0d1a55feebef4daa94a44673ecec33a7/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:38:55.291915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:55.306628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820abcc-c92d-4662-9b18-2b6f11c2ce5e admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:55.367475 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02963ae9-eaaa-4702-8afa-086f1d1aa367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:55.368375 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02963ae9-eaaa-4702-8afa-086f1d1aa367 admin admin] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:55.368584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02963ae9-eaaa-4702-8afa-086f1d1aa367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:55.368864 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02963ae9-eaaa-4702-8afa-086f1d1aa367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:55.369609 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02963ae9-eaaa-4702-8afa-086f1d1aa367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:55.371358 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02963ae9-eaaa-4702-8afa-086f1d1aa367 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:55.383895 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1436/2884] 10.222.0.46 () {58 vars in 1121 bytes} [Sat Jun 4 09:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 04 09:38:55.388498 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:38:55.399586 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f202b-53cf-478d-93a7-dd972584e28c admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:38:55.404445 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:38:55.405285 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:38:55.405752 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:38:55.406038 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] PATH_INFO: `/v3/projects/5e85e2e369fa4f9c921eeb851b45e21e/users/0d1a55feebef4daa94a44673ecec33a7/roles/c20defb0ae0a4e17b47a959da21cf17e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:38:55.408063 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e85e2e369fa4f9c921eeb851b45e21e, user_id=0d1a55feebef4daa94a44673ecec33a7, role_id=c20defb0ae0a4e17b47a959da21cf17e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:38:55.410958 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:38:55.423887 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1449/2885] 10.222.0.46 () {60 vars in 1482 bytes} [Sat Jun 4 09:38:55 2022] PUT /identity/v3/projects/5e85e2e369fa4f9c921eeb851b45e21e/users/0d1a55feebef4daa94a44673ecec33a7/roles/c20defb0ae0a4e17b47a959da21cf17e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:39:02.013421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:02.014304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:02.014304 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:02.045490 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] MFA Rules not processed for user `0d1a55feebef4daa94a44673ecec33a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:39:02.047233 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:02.129534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1437/2886] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:39:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:39:02.146527 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:02.148385 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:02.226650 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:02.253314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418230-8b6b-4b7c-ad7d-9171726dbe72 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:02.262969 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6e2f7-6017-4465-a144-3474cd8e2bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:02.263681 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e2f7-6017-4465-a144-3474cd8e2bcd service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:02.263976 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e2f7-6017-4465-a144-3474cd8e2bcd service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:02.264260 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e2f7-6017-4465-a144-3474cd8e2bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:02.348472 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6e2f7-6017-4465-a144-3474cd8e2bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:02.350170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6e2f7-6017-4465-a144-3474cd8e2bcd service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:02.361145 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1450/2887] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:39:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:39:12.126245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=65 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:12.493864 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=32 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:17.750735 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:17.752494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:17.866234 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dab095-5e85-4b0b-8740-3583b544c731 None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:17.871858 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:17.872607 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:17.872891 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:17.873168 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:17.875212 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:17.888528 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:17.890707 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:17.902333 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1438/2888] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:39:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:39:21.409114 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:21.409767 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:21.409884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:21.427806 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] MFA Rules not processed for user `c59c71bc492841b3b22c8e6d66599768`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:39:21.429073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:21.436369 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1451/2889] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:39:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:39:27.897117 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=41 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:31.422637 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=3 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:37.331037 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:37.333604 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:37.377269 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149638-3da0-4cc3-a322-85afd0de665f service placement] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:37.383459 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:37.383950 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:37.384235 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:37.384550 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:37.386620 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:37.401542 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:37.403172 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:37.416166 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1439/2890] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:39:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:39:41.252213 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:41.254048 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:41.327342 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ad2fb-124c-4109-842b-07e8016e9699 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:41.331727 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:41.332646 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:41.333080 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:41.333468 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:41.345228 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:41.346722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:41.360313 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1452/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:39:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:39:43.337817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:43.356336 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477b1d7-9595-4db7-b577-e9b7e881a434 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:43.362833 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:43.364782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:43.364782 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:43.365184 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:43.367636 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:43.460009 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:43.461557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:43.472929 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1440/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:39:48.019073 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:48.043264 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fde3d-6ab7-45d8-9158-dcad129d9e90 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:48.050587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:48.051557 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:48.051905 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:48.052252 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:48.054513 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:48.077965 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:48.080340 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:48.096118 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1453/2893] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:39:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:39:52.659479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:52.662310 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:52.680187 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624baaa-b19c-43bd-afee-e8be1a79f89a service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:52.686146 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:52.687001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:52.687287 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:52.687590 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] PATH_INFO: `/v3/users/79755c3f9eec4c5ab3e98038015bff32` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:52.689064 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79755c3f9eec4c5ab3e98038015bff32)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:52.691006 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:52.884596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79755c3f9eec4c5ab3e98038015bff32'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:52.885321 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79755c3f9eec4c5ab3e98038015bff32'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:52.898767 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79755c3f9eec4c5ab3e98038015bff32'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:52.921755 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79755c3f9eec4c5ab3e98038015bff32'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:52.922928 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79755c3f9eec4c5ab3e98038015bff32'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:53.004854 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1441/2894] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:39:52 2022] DELETE /identity/v3/users/79755c3f9eec4c5ab3e98038015bff32 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:39:53.272596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:53.298266 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a0852-59eb-4741-9796-686e69f6df21 service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:53.377604 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:53.378540 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:53.378852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:53.379125 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] PATH_INFO: `/v3/projects/d7bc412a2d2f4fc68c2a9f3562232807` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:53.380587 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bc412a2d2f4fc68c2a9f3562232807)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:53.382601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:53.511761 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bc412a2d2f4fc68c2a9f3562232807', 'request_id': 'req-ce990aa0-f118-49f9-8327-7a523b245374'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:53.523913 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bc412a2d2f4fc68c2a9f3562232807', 'request_id': 'req-ce990aa0-f118-49f9-8327-7a523b245374'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:53.605223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bc412a2d2f4fc68c2a9f3562232807', 'request_id': 'req-ce990aa0-f118-49f9-8327-7a523b245374'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:53.618748 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] The token cache is being invalidate because project d7bc412a2d2f4fc68c2a9f3562232807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:39:53.619044 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:53.619704 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:53.622699 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1454/2895] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:39:53 2022] DELETE /identity/v3/projects/d7bc412a2d2f4fc68c2a9f3562232807 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:39:53.626650 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:53.628975 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:53.652628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe4630-ff56-43e2-9cda-4aa32c01ff8a admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:53.711428 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:53.749887 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:53.754479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:53.754479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:53.754479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] PATH_INFO: `/v3/users/10cc3f8486704b77aeed38494dcf0408` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:53.754479 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10cc3f8486704b77aeed38494dcf0408)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:53.755109 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:54.032704 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10cc3f8486704b77aeed38494dcf0408'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.033395 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10cc3f8486704b77aeed38494dcf0408'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.042534 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10cc3f8486704b77aeed38494dcf0408'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.056443 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10cc3f8486704b77aeed38494dcf0408'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.057208 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10cc3f8486704b77aeed38494dcf0408'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.172331 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1442/2896] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:39:53 2022] DELETE /identity/v3/users/10cc3f8486704b77aeed38494dcf0408 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:39:54.561947 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:39:54.563889 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:39:54.583197 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce990aa0-f118-49f9-8327-7a523b245374 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:39:54.611519 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:39:54.658797 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:39:54.659665 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:39:54.659949 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:39:54.660233 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] PATH_INFO: `/v3/projects/8bfd730457a34f2192b1f1cf0d5a1aed` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:39:54.661554 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bfd730457a34f2192b1f1cf0d5a1aed)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:39:54.663592 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:39:54.796858 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bfd730457a34f2192b1f1cf0d5a1aed', 'request_id': 'req-9aabb0a9-722d-48cf-869e-4ba740cff873'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.805556 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bfd730457a34f2192b1f1cf0d5a1aed', 'request_id': 'req-9aabb0a9-722d-48cf-869e-4ba740cff873'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.868189 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bfd730457a34f2192b1f1cf0d5a1aed', 'request_id': 'req-9aabb0a9-722d-48cf-869e-4ba740cff873'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.881940 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] The token cache is being invalidate because project 8bfd730457a34f2192b1f1cf0d5a1aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:39:54.882230 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.883012 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:39:54.884722 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1455/2897] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:39:54 2022] DELETE /identity/v3/projects/8bfd730457a34f2192b1f1cf0d5a1aed => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:40:04.167840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=38,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:40:04.878204 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:41:39.557163 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:41:39.559654 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:41:39.627087 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860634d4-3a4c-40b3-b554-6eeb42bb1ca3 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:41:39.684960 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:41:39.685714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:41:39.686042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:41:39.686362 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:41:39.688710 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:41:39.793243 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:41:39.795106 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ca3d7-e6e6-48cd-afe0-19f8ec3d7430 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:41:39.807702 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1443/2898] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:41:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:41:41.066177 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:41:41.067832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:41:41.082170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabb0a9-722d-48cf-869e-4ba740cff873 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:41:41.086123 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:41:41.087138 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:41:41.087408 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:41:41.087692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:41:41.089710 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:41:41.104852 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:41:41.107187 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:41:41.120593 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1456/2899] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:41:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:41:44.747886 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] REQUEST_METHOD: `POST` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:41:44.748539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:41:44.748539 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:41:44.767840 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.auth.core [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] MFA Rules not processed for user `c59c71bc492841b3b22c8e6d66599768`. Rule list: `[]` (Enabled: `True`). {{(pid=80590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:41:44.769150 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:41:44.836244 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1444/2900] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 04 09:41:45.213503 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:41:45.229780 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b403771-ea5b-4855-ba5e-9e36f2077c35 service neutron] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:41:45.236168 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:41:45.237160 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:41:45.237694 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:41:45.238234 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:41:45.241753 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:41:45.260713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:41:45.263683 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d6a55-3e96-48e9-9b00-95c508dd25df service neutron] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:41:45.280333 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1457/2901] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:41:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:41:54.824835 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=99 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:41:55.272880 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=12 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:42:24.177563 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:42:24.189057 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6093476-e3a0-4613-943e-aa2226efa80d None None] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:42:24.193039 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:42:24.193708 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:42:24.193996 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:42:24.194276 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:42:24.196353 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:42:24.207663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:42:24.209183 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:42:24.219046 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1445/2902] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:42:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:42:34.214771 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=6 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:43:44.051122 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] REQUEST_METHOD: `POST` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:43:44.051755 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:43:44.052146 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:43:44.073223 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.auth.core [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] MFA Rules not processed for user `c59c71bc492841b3b22c8e6d66599768`. Rule list: `[]` (Enabled: `True`). {{(pid=80589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 04 09:43:44.074654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:43:44.082736 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1458/2903] 10.222.0.46 () {58 vars in 957 bytes} [Sat Jun 4 09:43:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 04 09:43:44.470660 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:43:44.481029 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111c730-aee1-48ce-8875-c2f32a566c9f service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:43:44.485586 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:43:44.486403 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:43:44.486788 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:43:44.487170 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:43:44.489520 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:43:44.501113 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:43:44.502584 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:43:44.513120 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1446/2904] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:43:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:43:46.342261 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:43:46.343713 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:43:46.432547 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f172a-54f2-4a95-a6b9-295f01d980b2 None None] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:43:46.438494 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:43:46.439372 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:43:46.439844 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:43:46.440239 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:43:46.442541 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:43:46.458698 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:43:46.460964 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:43:46.472933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1459/2905] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:43:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:43:53.934301 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:43:53.939514 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:43:53.974448 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5456f5-71a6-4884-9a58-8ea247569ba1 service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:43:53.983042 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:43:53.983800 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:43:53.984322 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:43:53.984632 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:43:53.986663 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:43:54.001776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:43:54.003496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:43:54.016949 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1447/2906] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:43:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:43:56.467832 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=35 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:44:01.693789 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:44:01.696481 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:44:01.715426 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66e35-e3da-4a5b-8014-f6ca80d5a6ae service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:44:01.721601 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:44:01.722272 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:44:01.722560 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:44:01.722845 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:44:01.724992 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:44:01.737625 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:44:01.739685 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:44:01.750479 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1460/2907] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:44:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:44:04.010557 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=12 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:44:05.018451 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:44:05.032001 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742de07-ba3f-47b2-8c49-c8eead24daa8 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:44:05.038012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c1d511f4c6341e09d54bf8709efdf8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:44:05.038817 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:44:05.039101 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:44:05.039484 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:44:05.041522 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:44:05.057012 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:44:05.058659 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:44:05.071323 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1448/2908] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:44:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:44:11.746279 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:44:15.065689 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=6 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:44:23.786093 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:44:23.799042 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4d28f-1d0e-4fa8-96e3-ff61473b9e95 service nova] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:44:23.804611 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:44:23.805308 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:44:23.805594 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:44:23.806036 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:44:23.807884 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:44:23.820696 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:44:23.822314 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:44:23.833697 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1461/2909] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:44:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:44:33.828692 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=6 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:45:48.613259 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:45:48.624634 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e9f89-6cd3-45a0-83fa-fe84325d8212 service nova] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:45:48.628596 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7837f3811b04542a6e7f56e7fa118d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:45:48.629275 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:45:48.629564 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:45:48.629839 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:45:48.631821 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:45:48.705496 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:45:48.707267 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:45:48.718831 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1449/2910] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 04 09:45:53.089596 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:45:53.109287 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa3135-8d81-40d5-8993-a792a892b04a service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:45:53.117830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:45:53.118654 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] REQUEST_METHOD: `GET` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:45:53.119029 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:45:53.119392 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:45:53.121830 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:45:53.136345 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:45:53.138100 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:45:53.151447 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1462/2911] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:45:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:45:57.943095 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:45:57.946533 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:45:57.968509 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0a15c-7190-4ab5-bc3c-1f42a2a672ae service neutron] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:45:57.976624 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:45:57.981256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:45:57.981256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:45:57.981256 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] PATH_INFO: `/v3/users/c59c71bc492841b3b22c8e6d66599768` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:45:57.981658 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59c71bc492841b3b22c8e6d66599768)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:45:57.985422 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:45:58.237030 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59c71bc492841b3b22c8e6d66599768'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:58.237717 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59c71bc492841b3b22c8e6d66599768'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:58.246776 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59c71bc492841b3b22c8e6d66599768'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:58.262553 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59c71bc492841b3b22c8e6d66599768'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:58.263265 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59c71bc492841b3b22c8e6d66599768'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:58.379324 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1450/2912] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:45:57 2022] DELETE /identity/v3/users/c59c71bc492841b3b22c8e6d66599768 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 04 09:45:58.712405 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:45:58.733572 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374c70-b5c3-4e12-9e0d-6c03a94b16f8 service placement] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:45:58.813589 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:45:58.814477 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:45:58.814762 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:45:58.815039 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] PATH_INFO: `/v3/projects/3c5c8695d627407f91f991dc91808a4b` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:45:58.816379 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5c8695d627407f91f991dc91808a4b)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:45:58.818729 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:45:58.965981 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5c8695d627407f91f991dc91808a4b', 'request_id': 'req-3447e910-03ce-424a-b23a-f6373d2c81d5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:58.974211 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5c8695d627407f91f991dc91808a4b', 'request_id': 'req-3447e910-03ce-424a-b23a-f6373d2c81d5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.047002 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5c8695d627407f91f991dc91808a4b', 'request_id': 'req-3447e910-03ce-424a-b23a-f6373d2c81d5'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.060933 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] The token cache is being invalidate because project 3c5c8695d627407f91f991dc91808a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:45:59.061280 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.062041 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.063873 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1463/2913] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:45:58 2022] DELETE /identity/v3/projects/3c5c8695d627407f91f991dc91808a4b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:45:59.069199 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:45:59.070837 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:45:59.086961 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ac2a-b8cf-4049-988b-113ec29655ab admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:45:59.168948 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:45:59.169915 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:45:59.170245 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:45:59.170628 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] PATH_INFO: `/v3/users/0d1a55feebef4daa94a44673ecec33a7` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:45:59.172239 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1a55feebef4daa94a44673ecec33a7)` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:45:59.174309 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:45:59.181125 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:45:59.463411 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1a55feebef4daa94a44673ecec33a7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.464238 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1a55feebef4daa94a44673ecec33a7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.472954 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1a55feebef4daa94a44673ecec33a7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.489676 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1a55feebef4daa94a44673ecec33a7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.490494 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.notifications [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1a55feebef4daa94a44673ecec33a7'} {{(pid=80590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:45:59.568409 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1451/2914] 10.222.0.46 () {58 vars in 1223 bytes} [Sat Jun 4 09:45:59 2022] DELETE /identity/v3/users/0d1a55feebef4daa94a44673ecec33a7 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:45:59.921932 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Authenticating user token {{(pid=80589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:45:59.924170 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.fernet_utils [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:45:59.947430 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447e910-03ce-424a-b23a-f6373d2c81d5 admin admin] Validating token access rules against request {{(pid=80589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:46:00.023336 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d4e8588f0b448199c8c5e35afe88ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbd4724a5a83452ab3b4b7b3823eccc9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:46:00.024217 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:46:00.024538 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:46:00.024816 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] PATH_INFO: `/v3/projects/5e85e2e369fa4f9c921eeb851b45e21e` {{(pid=80589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:46:00.026081 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e85e2e369fa4f9c921eeb851b45e21e)` {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:46:00.028085 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] RBAC: Authorization granted {{(pid=80589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:46:00.053021 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:46:00.135666 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e85e2e369fa4f9c921eeb851b45e21e', 'request_id': 'req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:46:00.142918 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e85e2e369fa4f9c921eeb851b45e21e', 'request_id': 'req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:46:00.224961 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e85e2e369fa4f9c921eeb851b45e21e', 'request_id': 'req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134'} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:46:00.239335 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] The token cache is being invalidate because project 5e85e2e369fa4f9c921eeb851b45e21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 04 09:46:00.239631 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:46:00.240352 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG keystone.notifications [None req-b586961c-e7e9-4cb7-acaf-f4a8bd01c134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 04 09:46:00.242003 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: [pid: 80589|app: 0|req: 1464/2915] 10.222.0.46 () {58 vars in 1232 bytes} [Sat Jun 4 09:45:59 2022] DELETE /identity/v3/projects/5e85e2e369fa4f9c921eeb851b45e21e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 04 09:46:09.563445 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:46:10.269237 dv-cinder-844695-1-iscsi devstack@keystone.service[80589]: DEBUG dbcounter [-] [80589] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 04 09:49:24.751452 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Authenticating user token {{(pid=80590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 04 09:49:24.753043 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:49:24.923808 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155efd-eb52-4e62-b325-268e82024772 admin admin] Validating token access rules against request {{(pid=80590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 04 09:49:24.992453 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa5724e78cc848a4af84a78bac03f64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '625af3032fbf42baa92a110d3363c3e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 04 09:49:24.993134 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] REQUEST_METHOD: `GET` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 04 09:49:24.993421 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] SCRIPT_NAME: `/identity` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 04 09:49:24.993714 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 04 09:49:24.995907 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.fernet_utils [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 04 09:49:25.009223 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 04 09:49:25.011069 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d2138-d405-49c5-814b-94ae854b16ae service placement] RBAC: Authorization granted {{(pid=80590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 04 09:49:25.022724 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: [pid: 80590|app: 0|req: 1452/2916] 10.222.0.46 () {60 vars in 1365 bytes} [Sat Jun 4 09:49:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 04 09:49:35.017811 dv-cinder-844695-1-iscsi devstack@keystone.service[80590]: DEBUG dbcounter [-] [80590] Writing DB stats keystone:SELECT=89 {{(pid=80590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}